Aggregate reads & writes in disk_io
#1205
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jakirkham
force-pushed
the
use_kvikio_futures
branch
4 times, most recently
from
e718585
to
aa6007e
Compare
Merged
jakirkham
added
improvement
jakirkham
changed the title
disk_iodisk_io
jakirkham
commented
Jun 28, 2023
jakirkham
force-pushed
the
use_kvikio_futures
branch
2 times, most recently
from
8071ccf
to
d72e98e
Compare
jakirkham
commented
Jun 28, 2023
jakirkham
commented
Jun 28, 2023
jakirkham
force-pushed
the
use_kvikio_futures
branch
2 times, most recently
from
de6806f
to
e2c24e6
Compare
Ensure that all buffers are backed by `uint8` NumPy arrays. This way they use hugepages when available and can be transferred to the GPU faster.
Before doing more expensive IO operations, allocate all memory needed in one go. This way it can be fed into the IO operations rapidly or be batched.
Simplify allocating memory for reading by using one `list`-comprehension for all cases.
Instead of writing each buffer one at a time, write all of them together in one call with `writelines`. This should result in fewer releases of the GIL and fewer kernel traps.
Handle reading and writing multiple buffers in one go with lower level OS primitives. Since Dask-CUDA is Linux only, can just rely on `readv` and `writev`. This way both reading *and* writing can be done in one call.
Instead of waiting on each `FutureIO` from KvikIO before creating the next one, submit all IO requests and then get each one at the end.
jakirkham
force-pushed
the
use_kvikio_futures
branch
from
e2c24e6
to
77a1600
Compare
jakirkham
force-pushed
the
use_kvikio_futures
branch
from
77a1600
to
7c7dafd
Compare
madsbk
reviewed
Jun 28, 2023
|
Thanks Mads! 🙏 Please let me know if anything else is needed here 🙂 |
pentschev
approved these changes
Jun 29, 2023
|
/merge |
|
Thank you both! 🙏 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow up to this discussion ( #925 (comment) )
uint8arrays for all host memory (benefits from hugepages on transfers)