eio_posix: use directory FDs instead of realpath #694
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SGrondin
reviewed
Feb 16, 2024
talex5
force-pushed
the
posix-dirfd
branch
5 times, most recently
from
71e3bf9 to
4261f87
Compare
realpath was an old hack from the libuv days.
|
The only noticeable change in the benchmarks is for the Path.stat rmtree time: Before: Removed in 20.94 s |
|
I've merged this so we can get some more testing. Please file issues with any problems you notice! |
talex5
added a commit
to talex5/opam-repository
that referenced
this pull request
Feb 22, 2024
CHANGES: New features: - eio_posix: use directory FDs instead of realpath (@talex5 ocaml-multicore/eio#694 ocaml-multicore/eio#696, reviewed by @SGrondin). Using realpath was an old hack from the libuv days and subject to races. It was also slow. - Keep pool of systhreads for blocking operations (@SGrondin @talex5 ocaml-multicore/eio#681). This is much faster than creating a new thread for each operation. It mainly benefits the eio_posix backend, as that uses lots of systhreads. - Make `Switch.on_release` thread-safe (@talex5 ocaml-multicore/eio#684, requested by @art-w and @clecat). This allows resource pools to be shared between domains easily. - Add `Eio.Path.read_link` (@talex5 ocaml-multicore/eio#686). - Add `Eio_unix.Fd.is_open` (@talex5 ocaml-multicore/eio#690). - Include backtrace in systhread errors (@talex5 ocaml-multicore/eio#688, reviewed by @SGrondin). Also, add `Eio.Exn.empty_backtrace` as a convenience. - eio.mock: add tracing support to mock backend (@talex5 ocaml-multicore/eio#687). - Improve tracing (@talex5 ocaml-multicore/eio#675 ocaml-multicore/eio#683 ocaml-multicore/eio#676, reviewed by @SGrondin). Update tracing section of README and trace more things (`run_in_systhread`, `close`, `submit`, `traceln`, cancellation and domain spawning). Documentation: - Link to verification work in docs (@talex5 ocaml-multicore/eio#682). - Add more trace diagrams to README (@talex5 ocaml-multicore/eio#698). - Adjust COC contacts (@polytypic ocaml-multicore/eio#685, reviewed by @Sudha247). Bug fixes: - eio_linux: retry `openat2` on `EAGAIN` (@talex5 ocaml-multicore/eio#693, reviewed by @SGrondin). - eio_posix and eio_windows: check for IO periodically (@talex5 ocaml-multicore/eio#674). - Handle EPERM when trying to initialise uring (@talex5 ocaml-multicore/eio#691). This can happen when using a Docker container. Build and tests: - Benchmark `Eio_unix.run_in_systhread` (@talex5 ocaml-multicore/eio#678, reviewed by @SGrondin). - Enable lintcstubs for `Eio_unix.Private` too (@talex5 ocaml-multicore/eio#689). - Stat benchmark: report cleanup time and optimise (@talex5 ocaml-multicore/eio#692). - Make benchmarks start faster (@talex5 ocaml-multicore/eio#673). - Update build for new eio-trace CLI (@talex5 ocaml-multicore/eio#699). - Expect opam-repo-ci tests to fail on macos (@talex5 ocaml-multicore/eio#672).
talex5
added a commit
to talex5/opam-repository
that referenced
this pull request
Feb 22, 2024
CHANGES: New features: - eio_posix: use directory FDs instead of realpath (@talex5 ocaml-multicore/eio#694 ocaml-multicore/eio#696, reviewed by @SGrondin). Using realpath was an old hack from the libuv days and subject to races. It was also slow. - Keep pool of systhreads for blocking operations (@SGrondin @talex5 ocaml-multicore/eio#681). This is much faster than creating a new thread for each operation. It mainly benefits the eio_posix backend, as that uses lots of systhreads. - Make `Switch.on_release` thread-safe (@talex5 ocaml-multicore/eio#684, requested by @art-w and @clecat). This allows resource pools to be shared between domains easily. - Add `Eio.Path.read_link` (@talex5 ocaml-multicore/eio#686). - Add `Eio_unix.Fd.is_open` (@talex5 ocaml-multicore/eio#690). - Include backtrace in systhread errors (@talex5 ocaml-multicore/eio#688, reviewed by @SGrondin). Also, add `Eio.Exn.empty_backtrace` as a convenience. - eio.mock: add tracing support to mock backend (@talex5 ocaml-multicore/eio#687). - Improve tracing (@talex5 ocaml-multicore/eio#675 ocaml-multicore/eio#683 ocaml-multicore/eio#676, reviewed by @SGrondin). Update tracing section of README and trace more things (`run_in_systhread`, `close`, `submit`, `traceln`, cancellation and domain spawning). Documentation: - Link to verification work in docs (@talex5 ocaml-multicore/eio#682). - Add more trace diagrams to README (@talex5 ocaml-multicore/eio#698). - Adjust COC contacts (@polytypic ocaml-multicore/eio#685, reviewed by @Sudha247). Bug fixes: - eio_linux: retry `openat2` on `EAGAIN` (@talex5 ocaml-multicore/eio#693, reviewed by @SGrondin). - eio_posix and eio_windows: check for IO periodically (@talex5 ocaml-multicore/eio#674). - Handle EPERM when trying to initialise uring (@talex5 ocaml-multicore/eio#691). This can happen when using a Docker container. Build and tests: - Benchmark `Eio_unix.run_in_systhread` (@talex5 ocaml-multicore/eio#678, reviewed by @SGrondin). - Enable lintcstubs for `Eio_unix.Private` too (@talex5 ocaml-multicore/eio#689). - Stat benchmark: report cleanup time and optimise (@talex5 ocaml-multicore/eio#692). - Make benchmarks start faster (@talex5 ocaml-multicore/eio#673). - Update build for new eio-trace CLI (@talex5 ocaml-multicore/eio#699). - Expect opam-repo-ci tests to fail on macos (@talex5 ocaml-multicore/eio#672).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
realpath was an old hack from the libuv days and isn't safe against races with symlinks being updated.
This should be faster too in some cases.