-
Notifications
You must be signed in to change notification settings - Fork 341
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
switch to async-io + blocking + multitask #836
Conversation
All tests pass here now |
Not sure what's wrong with ppc64 cross compilaion |
#837 just confirmed that the ppc64 failure is unrelated, not sure about the macos one: |
@Keruspe To figure out the crash on macOS, open |
Doesn't seem to give more info, but I think the ci files from master are used. I tried to temporary comment out ppc64 and the ppc64 tests were still run |
I created another project to toy with the CI.
The ubuntu and windows builders pass all the tests just fine. Apart from the ppc64, the other cross compilation targets pass fine too. I'm not really familiar with debugging macos issues so I don't really know where to start |
The first two commits work fine, the switch to multitask is the one causing the problem on macos |
Here are the errors I'm getting on my macOS machine: https://gist.github.com/stjepang/bbe3f07c947b2fd93a47b2e667f4428a |
@stjepang with this last commit, tests pass for OSX. Looks like |
Ok, instead of using the Not sure if there's a more elegant solution, something that we maybe want to fix on the async-io's side of things? |
Just opened smol-rs/async-io#4 as an alternative workaround |
Thanks for the PR to async-io, this is now fixed in async-io v0.1.4. |
Apart from the ppc64 cross compile which is broken even outside of this PR, everything should work fine now |
Looks good to me :) If all goes well, I think it’s time to stabilize crates async-io, blocking, and multitask. |
Rebased on top of master |
I think you can now remove |
It's still used if unstable is enabled and default disabled it seems? |
yes it is needed atm when there is no runtime at all imcluded |
Right, so we should be able to remove the dependency in the typical case. It pulls in a lot of transitive dependencies... |
@stjepang maybe the timers could live in their own crate? that way we could use them in the non runtime build, without pulling in anything else. |
@dignifiedquire Timers live in the |
I don't think there's a way to specify in Cargo.toml that a dep should be there only if an option is unset (wrt futures-timer). It doesn't seem to bring that much deps when wasm-bindgen isn't enabled though? |
Is the nightly fixed now? |
Should be |
With the switch to async-executor, the code is simpler, we gain more integration with smol 0.3, but the caveat is that tasks spawned onto the multi-threaded global executor cannot run local tasks anymore (since async-executor can only run the multi-threaded executor or the local executor on a thread but not both, so the runtime threads now only runs the global multi-threaded one) |
With something like smol-rs/async-executor#3 + reverting the |
What if we restrict That would simplify things because only |
Then I guess that's more or less what the current port to async-executor does? |
@stjepang What about returning an |
I expect in 99% cases people will just do |
Anything else holding this PR back? |
|
CI is just windows probably never tested without default and with unstable. |
and add a CI check for it Fixes async-rs#842 Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
Squashed those fixes into one commit and rebased |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm, and seems to be working as before
Something to be aware of in case you see issues related to Timers: smol-rs/async-io#6 |
way smol being divided in smaller crates??? |
Is there discussion on why smol / async-std was preferred to tokio for tremor? I'm trying to understand tremor as well as building some Rust async support for a cloud-native project. The info would be helpful. Thanks. |
smol now being divided in smaller crates, I tried porting async-std to those.
The async-io and blocking parts work fine and all tests seem to pass.
As of now, there are some tests hanging, namely because when we spawn a future from inside a block_on, it seems not to be polled, haven't figured out why yet.
The tokio enter function is directly copied from smol