-
Notifications
You must be signed in to change notification settings - Fork 118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support serenity simd_json #105
Conversation
vicky5124
commented
Dec 6, 2021
…erenity-rs#6) * Depend on Serenity's `next` branch on Songbird's `next` branch * Update the examples to use the `next` branch
Serenity's cache design has changed, this (finally) prevents the examples from failing to compile on CI.
Adds generics to any `Id` types on `Call`. Also includes the overlooked `Songbird::get_or_insert`. Closes serenity-rs#94. Tested using `cargo make ready`.
This matches a recent serenity change where `ClientBuilder` no longer has an explicit lifetime parameter. This was tested using `cargo make ready`.
modified: src/shards.rs
modified: src/handler.rs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR, just one minor nit regarding import formatting. I had a quick look at the repo and it seems that voice-model is untouched, so I think this is the extent of everything you need to change. :)
sorry! should have marked this as a draft, as there's still way more files to modify, and things that need to be fixed on serenity first before this can work and be merged. About the import order, cargo-fmt made it that way. |
modified: ../../../Cargo.toml modified: ../../../src/driver/mod.rs modified: ../../../src/input/dca.rs modified: ../../../src/input/error.rs modified: ../../../src/input/ffmpeg_src.rs modified: ../../../src/input/metadata.rs modified: ../../../src/input/ytdl_src.rs modified: ../../../src/ws.rs
modified: Cargo.toml modified: src/driver/mod.rs modified: src/input/dca.rs modified: src/input/error.rs modified: src/input/ffmpeg_src.rs modified: src/input/metadata.rs modified: src/input/ytdl_src.rs modified: src/ws.rs
modified: Cargo.toml modified: examples/serenity/voice_receive/Cargo.toml modified: src/input/metadata.rs modified: src/ws.rs
modified: examples/serenity/voice_receive/Cargo.toml
modified: README.md
modified: .github/workflows/ci.yml
modified: .github/workflows/ci.yml
new file: .cargo/config.toml
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently relies on serenity's simdjson support being fixed before I can end-to-end test this.
Otherwise compiles fine, comments are inline with respect to all driver-related JSON parsing.
.cargo/config.toml
Outdated
@@ -0,0 +1,2 @@ | |||
[build] | |||
rustflags = ["-C", "target-cpu=haswell"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not native
?
Can this also be moved to CI such that it's only invoked for the simd-json build task? I'm unsure if the spurious MacOS SIGILL originates from this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, unless you do CI with Arm, this target-cpu will work just fine.
Haswell was chosen over native as I've seen a few people mention it's less likely for it to fail CI due to "Illegal Instruction"; MacOS failed because there's still a chance it can fail from that, serenity has had this issue for quite a while now, and it's on Actions for it to be fixed.
Moving this to only the SIMD check would make it more likely to pass, and I'll do it when I have time for it, but it's hard to test CI when it's blocked behind it being accepted due to this being my first PR.
src/input/dca.rs
Outdated
serenity::json::prelude::from_slice::<DcaMetadata>(raw_json.as_mut_slice()) | ||
.map_err(DcaError::InvalidMetadata)? | ||
.into(); | ||
#[cfg(not(feature = "serenity"))] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
General feedback for all other driver/input uses: wouldn't it make more sense to use the simd-json library ourselves if that feature is selected? That way we can apply this to twilight too, and use serenity::json::*
only where we interface with serenity in the gateway portions. Twilight v0.8 now exposes their own JSON type via twilight_gateway::shard::raw_message::Message
from what I can see.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I somewhat planned for this, but I have 0 idea how twilight manages SIMD; I will not make a PR of something I'm completely unfamiliar with. If someone else wants to implement the SIMD feature for twilight, go for it.
Serenity has fixed it's SIMD issues, this PR is ready to be tested. |
Tried again there, still getting plenty of messages like below after a
|
37a15ec
to
e0ab49c
Compare
I think that was an issue in |
I'll manually fixup and test after #89 is merged (i.e., after a cleanup pass). Apologies that this has been left for quite so long. |
Okay, I've fixed this up to work without serenity now (and I believe JSON decode in DCA format and YoutubeDL handling should be accelerated too). This might take some fiddling to make sure the CI plays nice -- I've tried to make sure that RUSTFLAGS are only ever set on the Linux/simd target (since MacOS breaks routinely). |
This PR adds support for the simd-json library whenever decoding or encoding JSON responses. This may be enabled independently of serenity and twilight support for SIMD acceleration. Co-authored-by: Kyle Simpson <kyleandrew.simpson@gmail.com>
This PR adds support for the simd-json library whenever decoding or encoding JSON responses. This may be enabled independently of serenity and twilight support for SIMD acceleration. Co-authored-by: Kyle Simpson <kyleandrew.simpson@gmail.com>