What might porting std to use rustix look like?

[sunfishcode]

Porting Rust's std to use rustix instead of using libc directly for I/O on Unix-family platforms would factor out unsafe code, both for memory safety and I/O safety, and raw error handling, from std's main implementation code, making it easier to maintain. Secondarily, it'd also bring a number of optimizations, such as avoiding the allocation that std does on every system call that takes a string, and making system calls directly and avoiding errno.

cap-std and mustang demonstrate that rustix can do most of the I/O that std does, and the pieces that are missing should be straightforward to add. rustix and libc can coexist, so the work could be done incrementally.

rustix does depend on some things from std, though there's a working no_std branch, so we know the dependencies. The most interesting here are:

• std::os::raw::*, CStr, Ip*Addr, SocketAddr*, IoSlice* - Rust has avoided moving these into core because they invove OS-specific types, so perhaps we could introduce a new core_ffi library and move these into it? Or, perhaps rustix should define its own versions of these which convert to and from the std versions. rustix doesn't need them to be full-featured, so it should be doable.

• BorrowedFd, OwnedFd, etc. - Could these also make sense in a core_ffi library? Alternatively, rustix could similarly define its own.

• CString - Similarly, this could go into a new alloc_ffi library, or rustix could similarly define its own.

Once we have the dependencies sorted out, I imagine the next steps would be to make a temporary fork of std, and port some calls to use rustix, and then make a proposal to the std maintainers and use the fork to show how it simplifies the code and factors out unsafe, as the primary motivators.

For any of this to happen, I'll need help! Both because it's a lot of work, and because if there isn't community interest in helping with this, it's a sign that this doesn't actually make sense to do.

• Does the above sound reasonable? Is it missing anything major? Is there anything about rustix's design that should be different? What questions should I be asking here that I'm not?

• Are there people interested and available to implement pieces of this? Many of the pieces should be straightforward, and I'd be happy to mentor people and explain what needs to be done.

[sunfishcode]

rustix now has rustc-dep-of-std support, which includes definitions for OwnedFd, Ipv4Addr, SeekFrom, and a few other things rustix needs (for now), so it's no_std without all the dependencies, and it has the magic needed by std.

I've now created a rustc-dep-of-std branch to the rustix repo, which comments out rustix's optional dependencies, since building as a dependency of std seems to behave differently with respect to optional dependencies.

Building on these, I now have a fork of the rust tree which can start to use rustix in std! For example, see this patch, which factors out an unsafe and platform-specific buffer-length type logic.

To build it, set RUSTFLAGS="--cfg=linux_raw --cfg=asm --cfg=rustc_attrs" when running x.py.

If anyone is interested in helping out, here are the next steps (in either order):

• rustix in this mode defines IpAddr/Ipv4Addr/Ipv6Addr/SocketAddrV4/SocketAddrV6, as well as SeekFrom, IoSlice, and IoSliceMut. std should be updated to use rustix's versions instead of its own, similar to what's done for OwnedFd et al.
• Replace more libc calls with rustix calls, following the example! In theory, most things should be straightforward, as rustix aims to stay very close to the spirit of the libc calls.

If anyone takes a look at this and runs into trouble, or has questions, please feel free to post here, or reach out on zulip.

[lygstate]

Porting Rust's std to use rustix instead of using libc directly for I/O on Unix-family platforms would factor out unsafe code, both for memory safety and I/O safety, and raw error handling, from std's main implementation code, making it easier to maintain. Secondarily, it'd also bring a number of optimizations, such as avoiding the allocation that std does on every system call that takes a string, and making system calls directly and avoiding errno.

cap-std and mustang demonstrate that rustix can do most of the I/O that std does, and the pieces that are missing should be straightforward to add. rustix and libc can coexist, so the work could be done incrementally.

rustix does depend on some things from std, though there's a working no_std branch, so we know the dependencies. The most interesting here are:

• std::os::raw::*, CStr, Ip*Addr, SocketAddr*, IoSlice* - Rust has avoided moving these into core because they invove OS-specific types, so perhaps we could introduce a new core_ffi library and move these into it? Or, perhaps rustix should define its own versions of these which convert to and from the std versions. rustix doesn't need them to be full-featured, so it should be doable.

Rust already have ffi in core
https://github.com/rust-lang/rust/blob/master/library/core/src/ffi.rs
We can move part of these into https://github.com/rust-lang/rust/blob/master/library/core/src/ffi.rs
first.

• BorrowedFd, OwnedFd, etc. - Could these also make sense in a core_ffi library? Alternatively, rustix could similarly define its own.
• CString - Similarly, this could go into a new alloc_ffi library, or rustix could similarly define its own.

Once we have the dependencies sorted out, I imagine the next steps would be to make a temporary fork of std, and port some calls to use rustix, and then make a proposal to the std maintainers and use the fork to show how it simplifies the code and factors out unsafe, as the primary motivators.

For any of this to happen, I'll need help! Both because it's a lot of work, and because if there isn't community interest in helping with this, it's a sign that this doesn't actually make sense to do.

• Does the above sound reasonable? Is it missing anything major? Is there anything about rustix's design that should be different? What questions should I be asking here that I'm not?
• Are there people interested and available to implement pieces of this? Many of the pieces should be straightforward, and I'd be happy to mentor people and explain what needs to be done.

[sunfishcode]

rust-lang/rust#78802 is expected to allow SocketAddr and friends to be moved into core.

[lygstate]

According to https://github.com/rust-lang/rfcs/blob/master/text/2521-c_void-reunification.md#unresolved-questions, we can implement reset ctypes in core::ffi

[ikrivosheev]

@sunfishcode hello! Thank you for the great library! I would like to help with issue.

[sunfishcode]

@ikrivosheev Great! I guess the first step would be to see if you can build it; I posted instructions above, but I'd be interested to hear if it works for anyone other than me :-).

[sunfishcode]

I just made a pass through everything updating it to use the latest rustix and origin.

[sunfishcode]

Are there people interested and available to implement pieces of this?

It appears the answer is no; it seems no one else is interested in this at this time. If anyone is interested in working on this, or even in just talking about what this might look like, please reach out! I hope to keep minimally maintaining this code, as it's a useful testcase for rustix, but I don't currently have a plan beyond that.