This PR implements a custom scheduler for audio threads, which reduces thread use and (often) memory consumption.
To save threads and memory (e.g., packet buffer allocations), Songbird parks Mixer tasks which do not have any live Tracks.
These are now all co-located on a single async 'Idle' task.
This task is responsible for managing UDP keepalive messages for each task, maintaining event state, and executing any Mixer task messages.
Whenever any message arrives which adds a `Track`, the mixer task is moved to a live thread.
The Idle task inspects task counts and execution time on each thread, choosing the first live thread with room, and creating a new one if needed.
Each live thread is responsible for running as many live mixers as it can in a single tick every 20ms: this currently defaults to 16 mixers per thread, but is user-configurable.
A live thread also stores RTP packet blocks to be written into by each sub-task.
Each live thread has a conservative limit of 18ms that it will aim to stay under: if all work takes longer than this, it will offload the task with the highest mixing cost once per tick onto another (possibly new) live worker thread.
If a link such as [this](https://www.youtube.com/watch?v=ygY2qObZv24&list=RDygY2qObZv24)
is passed to `YoutubeDl` without the option, it would cause a deadlock
in my bot.
There were many videos where it produced a `Silent` packet first instead
of mixed or passthrough. The URL I added was one that produced a
passthrough packet so I used that. Please let me know if this is wrong.
Fixes the possibility of a spinlock while reading bytes from an async->sync adapter. This case can be observed with long (e.g., 10 hours of silence) videos which seem to be served slower than we would like to parse their headers.
The fix moves most communication to blocking: `read` calls first parse all messages form the async context in a non-blocking way, then swap to blocking if no bytes are available.
Tested using `cargo make ready` and "examples/serenity/voice" against the URL https://www.youtube.com/watch?v=g4mHPeMGTJM.
Converts any `io::ErrorKind::NotFound` from `TokioCommand` into a more useful string for users, e.g., `"could not find executable 'yt-dlp' on path"`.
Tested using `cargo make ready`.
Replaces the annoying dual-return (i.e., created `Call` *and* `Result<x>`) with a single `Return<Call/ConnectionInfo>`. Users are now informed via that a `Call` is created -- thus, cleanup in event of connection failure is now their responsibility.
Tested using `cargo make ready`.
Closes#65.
`youtube-dl` has a tendency to return mp4 files containing video and audio data vs `yt-dlp`. Naturally, the default stream in such cases will be the video data, so we fail to create a decoder.
This PR changes `LiveInput::promote` to first attempt to instantiate a decoder for the default track. If this fails to do so, then we reattempt this for all available tracks until one can be found. Previously, this method chose the default (or first available) track ID, and then failed if *that* track could not be decoded even if a compatible stream existed.
This was tested using `cargo make ready`, as well as manually verifying and adding a test case with a simple mp4 file.
Module docs mistakenly used the old doc-link format, so removing `create_player` never fired an error! These have also been partially rewritten to explain the role of `Track` and `TrackHandle`.
Includes some other misc fixes to links, mention of `TrackHandle::action` for metadata handling, etc.
Closes#140.
This PR adds support for the simd-json library whenever decoding or encoding JSON responses. This may be enabled independently of serenity and twilight support for SIMD acceleration.
Co-authored-by: Kyle Simpson <kyleandrew.simpson@gmail.com>
This extensive PR rewrites the internal mixing logic of the driver to use symphonia for parsing and decoding audio data, and rubato to resample audio. Existing logic to decode DCA and Opus formats/data have been reworked as plugins for symphonia. The main benefit is that we no longer need to keep yt-dlp and ffmpeg processes alive, saving a lot of memory and CPU: all decoding can be done in Rust! In exchange, we now need to do a lot of the HTTP handling and resumption ourselves, but this is still a huge net positive.
`Input`s have been completely reworked such that all default (non-cached) sources are lazy by default, and are no longer covered by a special-case `Restartable`. These now span a gamut from a `Compose` (lazy), to a live source, to a fully `Parsed` source. As mixing is still sync, this includes adapters for `AsyncRead`/`AsyncSeek`, and HTTP streams.
`Track`s have been reworked so that they only contain initialisation state for each track. `TrackHandles` are only created once a `Track`/`Input` has been handed over to the driver, replacing `create_player` and related functions. `TrackHandle::action` now acts on a `View` of (im)mutable state, and can request seeks/readying via `Action`.
Per-track event handling has also been improved -- we can now determine and propagate the reason behind individual track errors due to the new backend. Some `TrackHandle` commands (seek etc.) benefit from this, and now use internal callbacks to signal completion.
Due to associated PRs on felixmcfelix/songbird from avid testers, this includes general clippy tweaks, API additions, and other repo-wide cleanup. Thanks go out to the below co-authors.
Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com>
Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
This PR adds support for twilight v0.8, mainly adapting to significant API changes introduced by v0.7. As a result of these, twilight no longer accepts arbitrary JSON input, so it seemed sensible to adapt our `Shard` design to no longer require the same.
Adding to this, I've added in a trait to allow an arbitrary `Shard` to be installed, given only an implementation of a method to send a `VoiceStateUpdate`. Together, `Sharder::Generic` (songbird::shards::VoiceUpdate) and `Shard::Generic` (songbird::shards::GenericSharder) should allow any library to be hooked in to Songbird.
This PR was tested using `cargo make ready` and by manually testing `examples/twilight`.
This PR does the following:
* Changes both `Reader::Extension` and `Reader::ExtensionSeek` to use `symphonia::io::MediaSource`.
* Removes the `File` and `Vec` variants of readers, instead opting to provide a `from_file` and `from_memory` associated function to create readers from the `File` and `Cursor<Vec<u8>>` implementations of `MediaSource`.
* Removes the ReadSeek trait.
* Added a dependency on `symphonia_core`. This crate has no additional dependencies.
This change fixes tasks hanging due to rare cases of messages being lost between full Discord reconnections by placing a configurable timeout on the `ConnectionInfo` responses. This is a companion fix to [serenity#1255](https://github.com/serenity-rs/serenity/pull/1255). To make this doable, `Config`s are now used by all versions of `Songbird`/`Call`, and relevant functions are added to simplify setup with configuration. These are now non-exhaustive, correcting an earlier oversight. For future extensibility, this PR moves the return type of `join`/`join_gateway` into a custom future (no longer leaking flume's `RecvFut` type).
Additionally, this fixes the Makefile's feature sets for driver/gateway-only compilation.
This is a breaking change in:
* the return types of `join`/`join_gateway`
* moving `crate::driver::Config` -> `crate::Config`,
* `Config` and `JoinError` becoming `#[non_breaking]`.
This was tested via `cargo make ready`, and by testing `examples/serenity/voice_receive` with various timeout settings.
Adds support to the library for tokio 0.2 backward-compatibility. This should hopefully benefit, and prevent lavalink-rs from being blocked on this feature.
These can be reached using, e.g., `gateway-tokio-02`, `driver-tokio-02`, `serenity-rustls-tokio-02`, and `serenity-native-tokio-02` features.
Naturally, this requires some jiggering about with features and the underlying CI, which has been taken care of. Twilight can't be handled in this way, as their last tokio 0.2 version uses the deprecated Discord Gateway v6.
This should make dropping `ChildContainer`s and their parent `Input`s safer in async contexts.
It seems like SIGINT is insufficient to make wait terminate, but SIGKILL suffices. This introduced a new problem, namely that we have to remember and wait on *every* pid we create. This should, hopefully, put the issue of zombie processes to bed for good.
Linux/Unix requires that processes be waited, which is unfortunate as Windows lets us abandon them to the murderous whims of the OS. This PR adds Unix-specific behaviour to send a SIGINT before waiting on the process, and adds an additional thread per call for asset disposal on all platforms.
Closes#38.
---
* Close processes by SIGINT and wait on Unix
This seems to remedy the Linux-specific zombie processes. Addition of
nix as a dependency *should* be fine on Windows, since I believe it
compiles to an empty crate.
* Dispose of Tracks on auxiliary thread
This adds a mechanism for the mixer threads to perform potentially expensive deallocation/cleanup outside of the main loop, preventing deadline misses etc. This should make misbehaving `wait`s a bit more friendly.
This change is made with queue users in mind. Since sources
of this kind *know* how to (re)create themselves, they can
avoid being created at all until needed.
This also adds machinery to preload tracks *before* they are
needed, for gapless playback on queues and so on. Queues
make use of the event system to do this.
* Driver Benchmarks
Benchmarks driver use cases for single packet send,
multiple packet send, float vs opus, and the cost of
head-of-queue track removal.
Mix costs for large packet counts are also included.
This is a prelude to the optimisations discussed in
#21.
* Typo in benchmark
* Place Opus packet directly into packet buffer
Cleans up some other logic surrounding this, too. Gets a 16.9% perf improvement on opus packet passthrough (sub 5us here).
* Better track removal
In theory this should be faster, but it aint. Keeping in case
reducing struct sizes down the line magically makes this
faster.
* Reduce size of Input, TrackHandle
Metadata is now boxed away. Similarly, TrackHandles are neatly Arc'd to reduce their size to pointer length (and mitigate the impact of copies if we add in more fields).
Main goal: a lot of nested future/result folding.
This mainly modifies error handling for Tracks and TrackHandles to be
more consistent, and hides the underlying channel result passing in
get_info. Errors returned should be far clearer, and are domain
specific rather than falling back to a very opaque use of the underlying
channel error. It should be clearer to users why their handle commands
failed, or why they can't make a ytdl track loop or similar.
Also fixed/cleaned up Songbird::join(_gateway) to return in a single
await, sparing the user from the underlying channel details and repeated
Errs. I was trying for some time to extend the same graces to `Call`,
but could not figure out a sane way to get a 'static version of the
first future in the chain (i.e., the gateway send) so that the whole
thing could happen after dropping the lock around the Call. I really
wanted to fix this to happen as a single folded await too, but I think
this might need some crazy hack or redesign.
Far cleaner and more reliable than the old doc-link pattern. Also allowed me to spot some event types and sources which should have been made non_exhaustive.
Redresses a previous holdover from an attempt to get Restartable sources to work more neatly inside the synchronous mixer thread. This prevents `Restartable::*` from blocking without warning.
The initial fix at the time was to perform the restart work on a task provided by the tokio runtime as `executor::block_on` needs to be run from within a valid async runtime. Naturally, this completely missed the point that these closures should/could be async, without any need to fudge async functions into a sync wrapper.
Also removes the `From` for normal closures, as this will probably act as a footgun for folks on a single-threaded executor.
This implements a proof-of-concept for an improved audio frontend. The largest change is the introduction of events and event handling: both by time elapsed and by track events, such as ending or looping. Following on from this, the library now includes a basic, event-driven track queue system (which people seem to ask for unusually often). A new sample, `examples/13_voice_events`, demonstrates both the `TrackQueue` system and some basic events via the `~queue` and `~play_fade` commands.
Locks are removed from around the control of `Audio` objects, which should allow the backend to be moved to a more granular futures-based backend solution in a cleaner way.
