jacob/songbird
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Driver/Input: Migrate audio backend to Symphonia (#89)

Browse Source 
This extensive PR rewrites the internal mixing logic of the driver to use symphonia for parsing and decoding audio data, and rubato to resample audio. Existing logic to decode DCA and Opus formats/data have been reworked as plugins for symphonia. The main benefit is that we no longer need to keep yt-dlp and ffmpeg processes alive, saving a lot of memory and CPU: all decoding can be done in Rust! In exchange, we now need to do a lot of the HTTP handling and resumption ourselves, but this is still a huge net positive.

`Input`s have been completely reworked such that all default (non-cached) sources are lazy by default, and are no longer covered by a special-case `Restartable`. These now span a gamut from a `Compose` (lazy), to a live source, to a fully `Parsed` source. As mixing is still sync, this includes adapters for `AsyncRead`/`AsyncSeek`, and HTTP streams.

`Track`s have been reworked so that they only contain initialisation state for each track. `TrackHandles` are only created once a `Track`/`Input` has been handed over to the driver, replacing `create_player` and related functions. `TrackHandle::action` now acts on a `View` of (im)mutable state, and can request seeks/readying via `Action`.

Per-track event handling has also been improved -- we can now determine and propagate the reason behind individual track errors due to the new backend. Some `TrackHandle` commands (seek etc.) benefit from this, and now use internal callbacks to signal completion.

Due to associated PRs on felixmcfelix/songbird from avid testers, this includes general clippy tweaks, API additions, and other repo-wide cleanup. Thanks go out to the below co-authors.

Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com>
Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
This commit is contained in:
Kyle Simpson
2022-07-23 23:29:02 +01:00
parent 6c6ffa7ca8
commit 8cc7a22b0b
136 changed files with 9761 additions and 4891 deletions
Download Patch File Download Diff File Expand all files Collapse all files

406
src/tracks/mod.rs
View File

@@ -14,383 +14,189 @@
//! [`TrackHandle`]: struct.TrackHandle.html
//! [`create_player`]: fn.create_player.html
mod action;
mod command;
mod error;
mod handle;
mod looping;
mod mode;
mod queue;
mod ready;
mod state;
mod view;
pub use self::{command::*, error::*, handle::*, looping::*, mode::*, queue::*, state::*};
pub use self::{
action::*,
error::*,
handle::*,
looping::*,
mode::*,
queue::*,
ready::*,
state::*,
view::*,
};
pub(crate) use command::*;
use crate::{constants::*, driver::tasks::message::*, events::EventStore, input::Input};
use flume::{Receiver, TryRecvError};
use std::time::Duration;
use uuid::Uuid;
/// Control object for audio playback.
/// Initial state for audio playback.
///
/// Accessed by both commands and the playback code -- as such, access from user code is
/// almost always guarded via a [`TrackHandle`]. You should expect to receive
/// access to a raw object of this type via [`create_player`], for use in
/// [`Driver::play`] or [`Driver::play_only`].
/// [`Track`]s allow you to configure play modes, volume, event handlers, and other track state
/// before you pass an input to the [`Driver`].
///
/// Live track data is accesseed via a [`TrackHandle`], returned by [`Driver::play`] and
/// related methods.
///
/// # Example
///
/// ```rust,no_run
/// use songbird::{driver::Driver, ffmpeg, tracks::create_player};
/// use songbird::{driver::Driver, input::File, tracks::Track};
///
/// # async {
/// // A Call is also valid here!
/// let mut handler: Driver = Default::default();
/// let source = ffmpeg("../audio/my-favourite-song.mp3")
/// .await
/// .expect("This might fail: handle this error!");
/// let (mut audio, audio_handle) = create_player(source);
/// let source = File::new("../audio/my-favourite-song.mp3");
///
/// audio.set_volume(0.5);
///
/// handler.play_only(audio);
/// handler.play_only(Track::new(source.into()).volume(0.5));
///
/// // Future access occurs via audio_handle.
/// # };
/// ```
///
/// [`Driver::play_only`]: crate::driver::Driver::play_only
/// [`Driver`]: crate::driver::Driver
/// [`Driver::play`]: crate::driver::Driver::play
/// [`TrackHandle`]: TrackHandle
/// [`create_player`]: create_player
#[derive(Debug)]
pub struct Track {
/// Whether or not this sound is currently playing.
///
/// Can be controlled with [`play`] or [`pause`] if chaining is desired.
///
/// [`play`]: Track::play
/// [`pause`]: Track::pause
pub(crate) playing: PlayMode,
/// Defaults to [`PlayMode::Play`].
pub playing: PlayMode,
/// The desired volume for playback.
/// The volume for playback.
///
/// Sensible values fall between `0.0` and `1.0`.
/// Sensible values fall between `0.0` and `1.0`. Values outside this range can
/// cause clipping or other audio artefacts.
///
/// Can be controlled with [`volume`] if chaining is desired.
///
/// [`volume`]: Track::volume
pub(crate) volume: f32,
/// Defaults to `1.0`.
pub volume: f32,
/// Underlying data access object.
///
/// *Calling code is not expected to use this.*
pub(crate) source: Input,
/// The current playback position in the track.
pub(crate) position: Duration,
/// The total length of time this track has been active.
pub(crate) play_time: Duration,
/// The live or lazily-initialised audio stream to be played.
pub input: Input,
/// List of events attached to this audio track.
///
/// This may be used to add additional events to a track
/// before it is sent to the audio context for playing.
pub events: Option<EventStore>,
/// Channel from which commands are received.
///
/// Track commands are sent in this manner to ensure that access
/// occurs in a thread-safe manner, without allowing any external
/// code to lock access to audio objects and block packet generation.
pub(crate) commands: Receiver<TrackCommand>,
/// Handle for safe control of this audio track from other threads.
///
/// Typically, this is used by internal code to supply context information
/// to event handlers, though more may be cloned from this handle.
pub handle: TrackHandle,
/// Defaults to an empty set.
pub events: EventStore,
/// Count of remaining loops.
///
/// Defaults to play a track once (i.e., [`LoopState::Finite(0)`]).
///
/// [`LoopState::Finite(0)`]: LoopState::Finite
pub loops: LoopState,
/// Unique identifier for this track.
pub(crate) uuid: Uuid,
///
/// Defaults to a random 128-bit number.
pub uuid: Uuid,
}
impl Track {
/// Create a new track directly from an input, command source,
/// and handle.
///
/// In general, you should probably use [`create_player`].
///
/// [`create_player`]: fn.create_player.html
pub fn new_raw(source: Input, commands: Receiver<TrackCommand>, handle: TrackHandle) -> Self {
let uuid = handle.uuid();
/// Create a new track directly from an [`Input`] and a random [`Uuid`].
#[must_use]
pub fn new(input: Input) -> Self {
let uuid = Uuid::new_v4();
Self::new_with_uuid(input, uuid)
}
/// Create a new track directly from an [`Input`] with a custom [`Uuid`].
#[must_use]
pub fn new_with_uuid(input: Input, uuid: Uuid) -> Self {
Self {
playing: Default::default(),
playing: PlayMode::default(),
volume: 1.0,
source,
position: Default::default(),
play_time: Default::default(),
events: Some(EventStore::new_local()),
commands,
handle,
input,
events: EventStore::new_local(),
loops: LoopState::Finite(0),
uuid,
}
}
#[must_use]
/// Sets a track to playing if it is paused.
pub fn play(&mut self) -> &mut Self {
self.set_playing(PlayMode::Play)
}
/// Pauses a track if it is playing.
pub fn pause(&mut self) -> &mut Self {
self.set_playing(PlayMode::Pause)
}
/// Manually stops a track.
///
/// This will cause the audio track to be removed, with any relevant events triggered.
/// Stopped/ended tracks cannot be restarted.
pub fn stop(&mut self) -> &mut Self {
self.set_playing(PlayMode::Stop)
}
pub(crate) fn end(&mut self) -> &mut Self {
self.set_playing(PlayMode::End)
}
#[inline]
fn set_playing(&mut self, new_state: PlayMode) -> &mut Self {
self.playing = self.playing.change_to(new_state);
pub fn play(mut self) -> Self {
self.playing = PlayMode::Play;
self
}
/// Returns the current play status of this track.
pub fn playing(&self) -> PlayMode {
self.playing
#[must_use]
/// Pre-emptively pauses a track, preventing it from being automatically played.
pub fn pause(mut self) -> Self {
self.playing = PlayMode::Pause;
self
}
#[must_use]
/// Manually stops a track.
///
/// This will cause the audio track to be removed by the driver almost immediately,
/// with any relevant events triggered.
pub fn stop(mut self) -> Self {
self.playing = PlayMode::Stop;
self
}
#[must_use]
/// Sets [`volume`] in a manner that allows method chaining.
///
/// [`volume`]: Track::volume
pub fn set_volume(&mut self, volume: f32) -> &mut Self {
pub fn volume(mut self, volume: f32) -> Self {
self.volume = volume;
self
}
/// Returns the current playback position.
pub fn volume(&self) -> f32 {
self.volume
}
/// Returns the current playback position.
pub fn position(&self) -> Duration {
self.position
}
/// Returns the total length of time this track has been active.
pub fn play_time(&self) -> Duration {
self.play_time
}
#[must_use]
/// Set an audio track to loop a set number of times.
///
/// If the underlying [`Input`] does not support seeking,
/// then all calls will fail with [`TrackError::SeekUnsupported`].
///
/// [`Input`]: crate::input::Input
/// [`TrackError::SeekUnsupported`]: TrackError::SeekUnsupported
pub fn set_loops(&mut self, loops: LoopState) -> TrackResult<()> {
if self.source.is_seekable() {
self.loops = loops;
Ok(())
} else {
Err(TrackError::SeekUnsupported)
}
}
pub(crate) fn do_loop(&mut self) -> bool {
match self.loops {
LoopState::Infinite => true,
LoopState::Finite(0) => false,
LoopState::Finite(ref mut n) => {
*n -= 1;
true
},
}
}
/// Steps playback location forward by one frame.
pub(crate) fn step_frame(&mut self) {
self.position += TIMESTEP_LENGTH;
self.play_time += TIMESTEP_LENGTH;
}
/// Receives and acts upon any commands forwarded by TrackHandles.
///
/// *Used internally*, this should not be exposed to users.
pub(crate) fn process_commands(&mut self, index: usize, ic: &Interconnect) {
// Note: disconnection and an empty channel are both valid,
// and should allow the audio object to keep running as intended.
// Note that interconnect failures are not currently errors.
// In correct operation, the event thread should never panic,
// but it receiving status updates is secondary do actually
// doing the work.
loop {
match self.commands.try_recv() {
Ok(cmd) => {
use TrackCommand::*;
match cmd {
Play => {
self.play();
let _ = ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Mode(self.playing),
));
},
Pause => {
self.pause();
let _ = ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Mode(self.playing),
));
},
Stop => {
self.stop();
let _ = ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Mode(self.playing),
));
},
Volume(vol) => {
self.set_volume(vol);
let _ = ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Volume(self.volume),
));
},
Seek(time) =>
if let Ok(new_time) = self.seek_time(time) {
let _ = ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Position(new_time),
));
},
AddEvent(evt) => {
let _ = ic.events.send(EventMessage::AddTrackEvent(index, evt));
},
Do(action) => {
action(self);
let _ = ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Total(self.state()),
));
},
Request(tx) => {
let _ = tx.send(self.state());
},
Loop(loops) =>
if self.set_loops(loops).is_ok() {
let _ = ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Loops(self.loops, true),
));
},
MakePlayable => self.make_playable(),
}
},
Err(TryRecvError::Disconnected) => {
// this branch will never be visited.
break;
},
Err(TryRecvError::Empty) => {
break;
},
}
}
}
/// Ready a track for playing if it is lazily initialised.
///
/// Currently, only [`Restartable`] sources support lazy setup.
/// This call is a no-op for all others.
///
/// [`Restartable`]: crate::input::restartable::Restartable
pub fn make_playable(&mut self) {
self.source.reader.make_playable();
}
/// Creates a read-only copy of the audio track's state.
///
/// The primary use-case of this is sending information across
/// threads in response to a [`TrackHandle`].
///
/// [`TrackHandle`]: TrackHandle
pub fn state(&self) -> TrackState {
TrackState {
playing: self.playing,
volume: self.volume,
position: self.position,
play_time: self.play_time,
loops: self.loops,
}
}
/// Seek to a specific point in the track.
///
/// If the underlying [`Input`] does not support seeking,
/// then all calls will fail with [`TrackError::SeekUnsupported`].
///
/// [`Input`]: crate::input::Input
/// [`TrackError::SeekUnsupported`]: TrackError::SeekUnsupported
pub fn seek_time(&mut self, pos: Duration) -> TrackResult<Duration> {
if let Some(t) = self.source.seek_time(pos) {
self.position = t;
Ok(t)
} else {
Err(TrackError::SeekUnsupported)
}
pub fn loops(mut self, loops: LoopState) -> Self {
self.loops = loops;
self
}
#[must_use]
/// Returns this track's unique identifier.
pub fn uuid(&self) -> Uuid {
self.uuid
pub fn uuid(mut self, uuid: Uuid) -> Self {
self.uuid = uuid;
self
}
pub(crate) fn into_context(self) -> (TrackHandle, TrackContext) {
let (tx, receiver) = flume::unbounded();
let handle = TrackHandle::new(tx, self.uuid);
let context = TrackContext {
handle: handle.clone(),
track: self,
receiver,
};
(handle, context)
}
}
/// Creates a [`Track`] object to pass into the audio context, and a [`TrackHandle`]
/// for safe, lock-free access in external code.
///
/// Typically, this would be used if you wished to directly work on or configure
/// the [`Track`] object before it is passed over to the driver.
///
/// [`Track`]: Track
/// [`TrackHandle`]: TrackHandle
#[inline]
pub fn create_player(source: Input) -> (Track, TrackHandle) {
create_player_with_uuid(source, Uuid::new_v4())
}
/// Creates a [`Track`] and [`TrackHandle`] as in [`create_player`], allowing
/// a custom UUID to be set.
///
/// [`create_player`]: create_player
/// [`Track`]: Track
/// [`TrackHandle`]: TrackHandle
pub fn create_player_with_uuid(source: Input, uuid: Uuid) -> (Track, TrackHandle) {
let (tx, rx) = flume::unbounded();
let can_seek = source.is_seekable();
let metadata = source.metadata.clone();
let handle = TrackHandle::new(tx, can_seek, uuid, metadata);
let player = Track::new_raw(source, rx, handle.clone());
(player, handle)
/// Any [`Input`] (or struct which can be used as one) can also be made into a [`Track`].
impl<T: Into<Input>> From<T> for Track {
// NOTE: this is `Into` to support user-given structs which can
// only `impl Into<Input>`.
fn from(val: T) -> Self {
Track::new(val.into())
}
}