-
-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feature] io-uring support #92
Comments
First and foremost, it would have to implemented in lsm-tree (and value-log), fjall is just a wrapper around that. I'm not sure how feasible it is to rewrite everything into an async core, and if it really provides great performance improvements. And to benchmark, you have to actually migrate everything, which would be a lot of work. There are some benchmarks that don't look great: facebook/rocksdb#11017 But, there are certain isolated operations that may be optimizable using io-uring without having to migrate the entire code base, specifically:
There are situations where you don't need to wrap everything in spawn_blocking either. A rough guide is to not do work between two await points for more than 10-100µs.
But, when you write with fsync (consumer SSD: ~1ms, HDD: ~15ms) or read from an HDD (~15ms), you definitely want spawn_blocking, because you will wait for so long, you could perform other meaningful work while waiting. |
I see, thank you for the insightful response! So I guess my initial thought was if async io-uring would be added to fjall (lsm-tree), it would probably be best kept as a separate optional feature rather than a core rework, kind of like how sled provides a complementary |
Note that that function pretty much just moves the flush into a thread pool and awaits that:
The problem is you would have to basically double the entire library to have the sync and async implementations separately. |
I'm wondering if adding support for io-uring could be feasible without requiring any major rewrites, and if it's feasible, would there be enough benefits in doing so for async programs, instead of simply using spawn_blocking?
I could take a look at implementing something like this if it's deemed relevant enough.
The text was updated successfully, but these errors were encountered: