jacob/songbird
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Driver/Input: Migrate audio backend to Symphonia (#89)

Browse Source 
This extensive PR rewrites the internal mixing logic of the driver to use symphonia for parsing and decoding audio data, and rubato to resample audio. Existing logic to decode DCA and Opus formats/data have been reworked as plugins for symphonia. The main benefit is that we no longer need to keep yt-dlp and ffmpeg processes alive, saving a lot of memory and CPU: all decoding can be done in Rust! In exchange, we now need to do a lot of the HTTP handling and resumption ourselves, but this is still a huge net positive.

`Input`s have been completely reworked such that all default (non-cached) sources are lazy by default, and are no longer covered by a special-case `Restartable`. These now span a gamut from a `Compose` (lazy), to a live source, to a fully `Parsed` source. As mixing is still sync, this includes adapters for `AsyncRead`/`AsyncSeek`, and HTTP streams.

`Track`s have been reworked so that they only contain initialisation state for each track. `TrackHandles` are only created once a `Track`/`Input` has been handed over to the driver, replacing `create_player` and related functions. `TrackHandle::action` now acts on a `View` of (im)mutable state, and can request seeks/readying via `Action`.

Per-track event handling has also been improved -- we can now determine and propagate the reason behind individual track errors due to the new backend. Some `TrackHandle` commands (seek etc.) benefit from this, and now use internal callbacks to signal completion.

Due to associated PRs on felixmcfelix/songbird from avid testers, this includes general clippy tweaks, API additions, and other repo-wide cleanup. Thanks go out to the below co-authors.

Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com>
Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
This commit is contained in:
Kyle Simpson
2022-07-23 23:29:02 +01:00
parent 6c6ffa7ca8
commit 8cc7a22b0b
136 changed files with 9761 additions and 4891 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
src/constants.rs
View File

@@ -1,16 +1,16 @@
//! Constants affecting driver function and API handling.
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
use audiopus::{Bitrate, SampleRate};
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
use discortp::rtp::RtpType;
use std::time::Duration;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
/// The voice gateway version used by the library.
pub const VOICE_GATEWAY_VERSION: u8 = crate::model::constants::GATEWAY_VERSION;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
/// Sample rate of audio to be sent to Discord.
pub const SAMPLE_RATE: SampleRate = SampleRate::Hz48000;
@@ -23,10 +23,16 @@ pub const AUDIO_FRAME_RATE: usize = 50;
/// Length of time between any two audio frames.
pub const TIMESTEP_LENGTH: Duration = Duration::from_millis(1000 / AUDIO_FRAME_RATE as u64);
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
/// Default bitrate for audio.
pub const DEFAULT_BITRATE: Bitrate = Bitrate::BitsPerSecond(128_000);
/// Number of output samples at 48kHZ to produced when resampling subframes.
pub(crate) const RESAMPLE_OUTPUT_FRAME_SIZE: usize = MONO_FRAME_SIZE / 2;
/// The maximum number of bad frames to allow in an Opus source before blocking passthrough.
pub(crate) const OPUS_PASSTHROUGH_STRIKE_LIMIT: u8 = 3;
/// Number of samples in one complete frame of audio per channel.
///
/// This is equally the number of stereo (joint) samples in an audio frame.
@@ -70,6 +76,42 @@ pub const SILENT_FRAME: [u8; 3] = [0xf8, 0xff, 0xfe];
/// The one (and only) RTP version.
pub const RTP_VERSION: u8 = 2;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
/// Profile type used by Discord's Opus audio traffic.
pub const RTP_PROFILE_TYPE: RtpType = RtpType::Dynamic(120);
#[cfg(test)]
pub mod test_data {
/// URL for a source which YTDL must extract.
///
/// Referenced under CC BY-NC-SA 3.0 -- https://creativecommons.org/licenses/by-nc-sa/3.0/
pub const YTDL_TARGET: &str = "https://cloudkicker.bandcamp.com/track/94-days";
/// URL for a source which can be read via an Http Request.
///
/// Referenced under CC BY-NC-SA 3.0 -- https://creativecommons.org/licenses/by-nc-sa/3.0/
pub const HTTP_TARGET: &str = "https://github.com/FelixMcFelix/songbird/raw/symphonia/resources/Cloudkicker%20-%202011%2007.mp3";
/// URL for an opus/ogg source which can be read via an Http Request.
///
/// Referenced under CC BY 3.0 -- https://creativecommons.org/licenses/by/3.0/
pub const HTTP_OPUS_TARGET: &str = "https://github.com/FelixMcFelix/songbird/raw/symphonia/resources/Cloudkicker%20-%20Making%20Will%20Mad.opus";
/// URL for an opus/webm source which can be read via an Http Request.
///
/// Referenced under CC BY 3.0 -- https://creativecommons.org/licenses/by/3.0/
pub const HTTP_WEBM_TARGET: &str = "https://github.com/FelixMcFelix/songbird/raw/symphonia/resources/Cloudkicker%20-%20Making%20Will%20Mad.webm";
/// Path to a DCA source.
///
/// Referenced under CC BY-NC-SA 3.0 -- https://creativecommons.org/licenses/by-nc-sa/3.0/
pub const FILE_DCA_TARGET: &str = "resources/Cloudkicker - 2011 07.dca1";
/// Path to an opus source which can be read via a File.
///
/// Referenced under CC BY 3.0 -- https://creativecommons.org/licenses/by/3.0/
pub const FILE_WEBM_TARGET: &str = "resources/Cloudkicker - Making Will Mad.webm";
/// Path to a Wav source which can be read via a File.
pub const FILE_WAV_TARGET: &str = "resources/loop.wav";
}