Driver/Input: Migrate audio backend to Symphonia (#89)

This extensive PR rewrites the internal mixing logic of the driver to use symphonia for parsing and decoding audio data, and rubato to resample audio. Existing logic to decode DCA and Opus formats/data have been reworked as plugins for symphonia. The main benefit is that we no longer need to keep yt-dlp and ffmpeg processes alive, saving a lot of memory and CPU: all decoding can be done in Rust! In exchange, we now need to do a lot of the HTTP handling and resumption ourselves, but this is still a huge net positive.

`Input`s have been completely reworked such that all default (non-cached) sources are lazy by default, and are no longer covered by a special-case `Restartable`. These now span a gamut from a `Compose` (lazy), to a live source, to a fully `Parsed` source. As mixing is still sync, this includes adapters for `AsyncRead`/`AsyncSeek`, and HTTP streams.

`Track`s have been reworked so that they only contain initialisation state for each track. `TrackHandles` are only created once a `Track`/`Input` has been handed over to the driver, replacing `create_player` and related functions. `TrackHandle::action` now acts on a `View` of (im)mutable state, and can request seeks/readying via `Action`.

Per-track event handling has also been improved -- we can now determine and propagate the reason behind individual track errors due to the new backend. Some `TrackHandle` commands (seek etc.) benefit from this, and now use internal callbacks to signal completion.

Due to associated PRs on felixmcfelix/songbird from avid testers, this includes general clippy tweaks, API additions, and other repo-wide cleanup. Thanks go out to the below co-authors.

Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com>
Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
This commit is contained in:
Kyle Simpson
2022-07-23 23:29:02 +01:00
parent 6c6ffa7ca8
commit 8cc7a22b0b
136 changed files with 9761 additions and 4891 deletions

View File

@@ -0,0 +1,146 @@
use crate::input::AudioStreamError;
use audiopus::error::Error as OpusError;
use serde_json::Error as JsonError;
use std::{
error::Error as StdError,
fmt::{Display, Formatter, Result as FmtResult},
};
use streamcatcher::CatcherError;
use symphonia_core::errors::Error as SymphError;
use tokio::task::JoinError;
/// Errors encountered using a [`Memory`] cached source.
///
/// [`Memory`]: super::Memory
#[derive(Debug)]
pub enum Error {
/// The audio stream could not be created.
Create(AudioStreamError),
/// The audio stream failed to be created due to a panic in `spawn_blocking`.
CreatePanicked,
/// Streamcatcher's configuration was illegal, and the cache could not be created.
Streamcatcher(CatcherError),
/// The input stream had already been read (i.e., `Parsed`) and so the whole stream
/// could not be used.
StreamNotAtStart,
}
impl Display for Error {
fn fmt(&self, f: &mut Formatter<'_>) -> FmtResult {
match self {
Self::Create(c) => f.write_fmt(format_args!("failed to create audio stream: {}", c)),
Self::CreatePanicked => f.write_str("sync thread panicked while creating stream"),
Self::Streamcatcher(s) =>
f.write_fmt(format_args!("illegal streamcatcher config: {}", s)),
Self::StreamNotAtStart =>
f.write_str("stream cannot have been pre-read/parsed, missing headers"),
}
}
}
impl StdError for Error {}
impl From<AudioStreamError> for Error {
fn from(val: AudioStreamError) -> Self {
Self::Create(val)
}
}
impl From<CatcherError> for Error {
fn from(val: CatcherError) -> Self {
Self::Streamcatcher(val)
}
}
impl From<JoinError> for Error {
fn from(_val: JoinError) -> Self {
Self::CreatePanicked
}
}
/// Errors encountered using a [`Compressed`] or [`Decompressed`] cached source.
///
/// [`Compressed`]: super::Compressed
/// [`Decompressed`]: super::Decompressed
#[derive(Debug)]
pub enum CodecCacheError {
/// The audio stream could not be created.
Create(AudioStreamError),
/// Symphonia failed to parse the container or decode the default stream.
Parse(SymphError),
/// The Opus encoder could not be created.
Opus(OpusError),
/// The file's metadata could not be converted to JSON.
MetadataEncoding(JsonError),
/// The input's metadata was too large after conversion to JSON to fit in a DCA file.
MetadataTooLarge,
/// The audio stream failed to be created due to a panic in `spawn_blocking`.
CreatePanicked,
/// The audio stream's channel count could not be determined.
UnknownChannelCount,
/// Streamcatcher's configuration was illegal, and the cache could not be created.
Streamcatcher(CatcherError),
/// The input stream had already been read (i.e., `Parsed`) and so the whole stream
/// could not be used.
StreamNotAtStart,
}
impl Display for CodecCacheError {
fn fmt(&self, f: &mut Formatter<'_>) -> FmtResult {
match self {
Self::Create(c) => f.write_fmt(format_args!("failed to create audio stream: {}", c)),
Self::Parse(p) => f.write_fmt(format_args!("failed to parse audio format: {}", p)),
Self::Opus(o) => f.write_fmt(format_args!("failed to create Opus encoder: {}", o)),
Self::MetadataEncoding(m) => f.write_fmt(format_args!(
"failed to convert track metadata to JSON: {}",
m
)),
Self::MetadataTooLarge => f.write_str("track metadata was too large, >= 32kiB"),
Self::CreatePanicked => f.write_str("sync thread panicked while creating stream"),
Self::UnknownChannelCount =>
f.write_str("audio stream's channel count could not be determined"),
Self::Streamcatcher(s) =>
f.write_fmt(format_args!("illegal streamcatcher config: {}", s)),
Self::StreamNotAtStart =>
f.write_str("stream cannot have been pre-read/parsed, missing headers"),
}
}
}
impl StdError for CodecCacheError {}
impl From<AudioStreamError> for CodecCacheError {
fn from(val: AudioStreamError) -> Self {
Self::Create(val)
}
}
impl From<CatcherError> for CodecCacheError {
fn from(val: CatcherError) -> Self {
Self::Streamcatcher(val)
}
}
impl From<JoinError> for CodecCacheError {
fn from(_val: JoinError) -> Self {
Self::CreatePanicked
}
}
impl From<JsonError> for CodecCacheError {
fn from(val: JsonError) -> Self {
Self::MetadataEncoding(val)
}
}
impl From<OpusError> for CodecCacheError {
fn from(val: OpusError) -> Self {
Self::Opus(val)
}
}
impl From<SymphError> for CodecCacheError {
fn from(val: SymphError) -> Self {
Self::Parse(val)
}
}