-
Notifications
You must be signed in to change notification settings - Fork 13.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
STD support for the ESP-IDF framework #87666
Conversation
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @dtolnay (or someone else) soon. Please see the contribution instructions for more information. |
|
r? @rust-lang/libs |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
c7966fd
to
b1e8daa
Compare
Note that as part of only supporting fallible allocation in the linux kernel (rust-lang/rfcs#3140) there is ongoing discussion around making std features optional and how to support that on the cargo side. rust-lang/rfcs#3146 I'm not sure what exactly the roadmap on those parts are but it sounds like it would be appropriate here if you only want to support a subset of the standard library in the beginning. But putting whole modules behind feature gates that have yet to be introduced instead of stubbing them could require some work. |
That's very useful info, thanks for sharing! My problem with it is a little bit that this patch set has been in development since approx last November, and waiting for another Cargo nightly feature to first of all hit nightly, and then suffering through its stabilization pains, change of semantics and so on might be a bit too much. Especially considering that we are already sailing in an ocean of Cargo unstable features, that - when changed or regressed - did hit us in the past:
So I would rather take a conservative approach here and stub what is not supported, if there is no strong, well-grounded objection from the Rust maintainers. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
☔ The latest upstream changes (presumably #87535) made this pull request unmergeable. Please resolve the merge conflicts. |
This is fine as long as the underlying implementation is lock-free (doesn't use a lock internally to emulate atomics). The requirement here is that a signal/interrupt handler should never deadlock if it tries to use an Generally this is implemented by disabling interrupt handlers or by having the interrupt handler restart the atomic sequence in the interrupted thread upon returning. We already have something similar for ARMv4 on Linux (atomic instructions were introduced in ARMv6) where support for atomic operations is provided by the kernel: https://github.com/rust-lang/compiler-builtins/blob/master/src/arm_linux.rs |
Now that libc
|
@bors r+ |
📌 Commit 459eaa6 has been approved by |
☀️ Test successful - checks-actions |
Pkgsrc changes: * Remove one now-longer-applicable patch, adjust a few others * Bump bootstrap requirements to 1.55.0. Upstream changes: Version 1.56.0 (2021-10-21) ======================== Language -------- - [The 2021 Edition is now stable.][rust#88100] See [the edition guide][rust-2021-edition-guide] for more details. - [The pattern in `binding @ pattern` can now also introduce new bindings.] [rust#85305] - [Union field access is permitted in `const fn`.][rust#85769] [rust-2021-edition-guide]: https://doc.rust-lang.org/nightly/edition-guide/rust-2021/index.html Compiler -------- - [Upgrade to LLVM 13.][rust#87570] - [Support memory, address, and thread sanitizers on aarch64-unknown-freebsd.][rust#88023] - [Allow specifying a deployment target version for all iOS targets][rust#87699] - [Warnings can be forced on with `--force-warn`.][rust#87472] This feature is primarily intended for usage by `cargo fix`, rather than end users. - [Promote `aarch64-apple-ios-sim` to Tier 2\*.][rust#87760] - [Add `powerpc-unknown-freebsd` at Tier 3\*.][rust#87370] - [Add `riscv32imc-esp-espidf` at Tier 3\*.][rust#87666] \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Allow writing of incomplete UTF-8 sequences via stdout/stderr on Windows.] [rust#83342] The Windows console still requires valid Unicode, but this change allows splitting a UTF-8 character across multiple write calls. This allows, for instance, programs that just read and write data buffers (e.g. copying a file to stdout) without regard for Unicode or character boundaries. - [Prefer `AtomicU{64,128}` over Mutex for Instant backsliding protection.] [rust#83093] For this use case, atomics scale much better under contention. - [Implement `Extend<(A, B)>` for `(Extend<A>, Extend<B>)`][rust#85835] - [impl Default, Copy, Clone for std::io::Sink and std::io::Empty][rust#86744] - [`impl From<[(K, V); N]>` for all collections.][rust#84111] - [Remove `P: Unpin` bound on impl Future for Pin.][rust#81363] - [Treat invalid environment variable names as non-existent.][rust#86183] Previously, the environment functions would panic if given a variable name with an internal null character or equal sign (`=`). Now, these functions will just treat such names as non-existent variables, since the OS cannot represent the existence of a variable with such a name. Stabilised APIs --------------- - [`std::os::unix::fs::chroot`] - [`UnsafeCell::raw_get`] - [`BufWriter::into_parts`] - [`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`] These APIs were previously stable in `std`, but are now also available in `core`. - [`Vec::shrink_to`] - [`String::shrink_to`] - [`OsString::shrink_to`] - [`PathBuf::shrink_to`] - [`BinaryHeap::shrink_to`] - [`VecDeque::shrink_to`] - [`HashMap::shrink_to`] - [`HashSet::shrink_to`] These APIs are now usable in const contexts: - [`std::mem::transmute`] - [`[T]::first`][`slice::first`] - [`[T]::split_first`][`slice::split_first`] - [`[T]::last`][`slice::last`] - [`[T]::split_last`][`slice::split_last`] Cargo ----- - [Cargo supports specifying a minimum supported Rust version in Cargo.toml.] [`rust-version`] This has no effect at present on dependency version selection. We encourage crates to specify their minimum supported Rust version, and we encourage CI systems that support Rust code to include a crate's specified minimum version in the text matrix for that crate by default. Compatibility notes ------------------- - [Update to new argument parsing rules on Windows.][rust#87580] This adjusts Rust's standard library to match the behavior of the standard libraries for C/C++. The rules have changed slightly over time, and this PR brings us to the latest set of rules (changed in 2008). - [Disallow the aapcs calling convention on aarch64][rust#88399] This was already not supported by LLVM; this change surfaces this lack of support with a better error message. - [Make `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` warn by default][rust#87385] - [Warn when an escaped newline skips multiple lines.][rust#87671] - [Calls to `libc::getpid` / `std::process::id` from `Command::pre_exec` may return different values on glibc <= 2.24.][rust#81825] Rust now invokes the `clone3` system call directly, when available, to use new functionality available via that system call. Older versions of glibc cache the result of `getpid`, and only update that cache when calling glibc's clone/fork functions, so a direct system call bypasses that cache update. glibc 2.25 and newer no longer cache `getpid` for exactly this reason. Internal changes ---------------- These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools. - [LLVM is compiled with PGO in published x86_64-unknown-linux-gnu artifacts.][rust#88069] This improves the performance of most Rust builds. - [Unify representation of macros in internal data structures.][rust#88019] This change fixes a host of bugs with the handling of macros by the compiler, as well as rustdoc. [`std::os::unix::fs::chroot`]: https://doc.rust-lang.org/stable/std/os/unix/fs/fn.chroot.html [`Iterator::intersperse`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse [`Iterator::intersperse_with`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse [`UnsafeCell::raw_get`]: https://doc.rust-lang.org/stable/std/cell/struct.UnsafeCell.html#method.raw_get [`BufWriter::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.BufWriter.html#method.into_parts [`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]: rust-lang/rust#84662 [`Vec::shrink_to`]: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.shrink_to [`String::shrink_to`]: https://doc.rust-lang.org/stable/std/string/struct.String.html#method.shrink_to [`OsString::shrink_to`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.shrink_to [`PathBuf::shrink_to`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.shrink_to [`BinaryHeap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.shrink_to [`VecDeque::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.VecDeque.html#method.shrink_to [`HashMap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_map/struct.HashMap.html#method.shrink_to [`HashSet::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_set/struct.HashSet.html#method.shrink_to [`std::mem::transmute`]: https://doc.rust-lang.org/stable/std/mem/fn.transmute.html [`slice::first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.first [`slice::split_first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_first [`slice::last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.last [`slice::split_last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_last [`rust-version`]: https://doc.rust-lang.org/nightly/cargo/reference/manifest.html#the-rust-version-field [rust#87671]: rust-lang/rust#87671 [rust#86183]: rust-lang/rust#86183 [rust#87385]: rust-lang/rust#87385 [rust#88100]: rust-lang/rust#88100 [rust#86860]: rust-lang/rust#86860 [rust#84039]: rust-lang/rust#84039 [rust#86492]: rust-lang/rust#86492 [rust#88363]: rust-lang/rust#88363 [rust#85305]: rust-lang/rust#85305 [rust#87832]: rust-lang/rust#87832 [rust#88069]: rust-lang/rust#88069 [rust#87472]: rust-lang/rust#87472 [rust#87699]: rust-lang/rust#87699 [rust#87570]: rust-lang/rust#87570 [rust#88023]: rust-lang/rust#88023 [rust#87760]: rust-lang/rust#87760 [rust#87370]: rust-lang/rust#87370 [rust#87580]: rust-lang/rust#87580 [rust#83342]: rust-lang/rust#83342 [rust#83093]: rust-lang/rust#83093 [rust#88177]: rust-lang/rust#88177 [rust#88548]: rust-lang/rust#88548 [rust#88551]: rust-lang/rust#88551 [rust#88299]: rust-lang/rust#88299 [rust#88220]: rust-lang/rust#88220 [rust#85835]: rust-lang/rust#85835 [rust#86879]: rust-lang/rust#86879 [rust#86744]: rust-lang/rust#86744 [rust#84662]: rust-lang/rust#84662 [rust#86593]: rust-lang/rust#86593 [rust#81050]: rust-lang/rust#81050 [rust#81363]: rust-lang/rust#81363 [rust#84111]: rust-lang/rust#84111 [rust#85769]: rust-lang/rust#85769 (comment) [rust#88490]: rust-lang/rust#88490 [rust#88269]: rust-lang/rust#88269 [rust#84176]: rust-lang/rust#84176 [rust#88399]: rust-lang/rust#88399 [rust#88227]: rust-lang/rust#88227 [rust#88200]: rust-lang/rust#88200 [rust#82776]: rust-lang/rust#82776 [rust#88077]: rust-lang/rust#88077 [rust#87728]: rust-lang/rust#87728 [rust#87050]: rust-lang/rust#87050 [rust#87619]: rust-lang/rust#87619 [rust#81825]: rust-lang/rust#81825 (comment) [rust#88019]: rust-lang/rust#88019 [rust#87666]: rust-lang/rust#87666
Dear all,
This PR is implementing libStd support for the ESP-IDF newlib-based framework, which is the open source SDK provided by Espressif for their MCU family (esp32, esp32s2, esp32c3 and all other forthcoming ones).
Note that this PR has a sibling PR against the libc crate, which implements proper declarations for all ESP-IDF APIs which are necessary for libStd support.
Implementation approach
The ESP-IDF framework - despite being bare metal - offers a relatively complete POSIX API based on newlib.
pthread
, BSD sockets, file descriptors, and even a small file-system VFS layer. Perhaps the only significant exception is the lack of support for processes, which is to be expected of course on bare metal.Therefore, the libStd support is implemented as a set of (hopefully small) changes to the
sys/unix
family of modules, in the form of conditional-compilation branches based either ontarget_os = "espidf"
or in a couple of cases - based ontarget_env = "newlib"
(the latter was already there actually and is not part of this patch).The PR also contains two new targets:
riscv32imc-esp-espidf
riscv32imac-esp-espidf
... which are essentially copies of
riscv32imc-unknown-none-elf
andriscv32imac-unknown-none-elf
, but enriched with properlinker
,linker_flavor
,families
,os
,env
etc. specifications so that (a) the proper conditional compilation branches in libStd are selected when compiling with these targets and (b) the correct linker is used.Since support for atomics is a precondition for libStd, the
riscv32imc-esp-espidf
target additionally is configured in such a way, so as to emit libcalls to the__sync*
&__atomic*
GCC functions, which are already implemented in the ESP-IDF framework. If this modification is not acceptable, we can also live with only theriscv32imac-esp-espidf
target as well. While the RiscV chips of Espressif lack native atomics support, the relevant instructions are transparently emulated in the ESP-IDF framework using invalid instruction trap. This modification was implemented specifically with Rust support in mind.Target maintainers
In case this PR eventually gets merged, you can list myself as a Target Maintainer.
More importantly, Espressif (the chip vendor) is now actively involved and embracing all Rust-related efforts which were originally a community effort. In light of that, I suppose @MabezDev - who initiated the Rust-on-Espressif efforts back in time and who now works for Espressif won't object to being listed as a maintainer as well.
EDIT: I was hinted (thanks, @Urgau) that answering the Tier 3 policy explicitly might be helpful. Answers below.
Tier 3 Target Policy - answers
Hopefully, the changes introduced by the ESP-IDF libStd support are rather on the small side. They are completely contained within the
sys/unix
set of modules (that is, aside from the obviously necessary one-liners in theunwind
crate and inbuild.rs
).@ivmarkov
@MabezDev
The two introduced targets follow as much as possible the naming conventions of the other targets. I.e. taking the bare-metal
riscv32imac_unknown_none_elf
as a base:none
withespidf
to designate thetarget_os
._elf
was removed, as the non-bare metal targets seem not to have it-newlib
was deliberately NOT added at the end, as I believe the chance of having two simultaneously active separate targets for the ESP-IDF framework with different C libraries (say, newlib vs musl) is way too smallunknown
withesp
which is kind of the name of the whole chipset MCU family (and abbreviation from Espressif which is too long). It will stayesp
for all RiscV32-based MCUs of the company, as they all use the riscv32imc instruction set. By necessity however (disambiguation), it will beesp32
oresp32s2
oresp32s3
for the Xtensa-based MCUs as all of these have their own variation of the Xtensa architecture. (The Xtensa targets are not part of this PR, even though they would use 1:1 the same LibStd implementation provided here, as they depend on the upstreaming of the Xtensa architecture support in LLVM; this upstreaming this is currently in progress.)There was also a preceding discussion on the topic here.
We are explicitly putting an
-espidf
suffix to designate that the target is specifically for Rust + ESP-IDFAgreed.
To the best of our knowledge, it doesn't.
MIT + Apache 2.0
Requirements are not changed for any other target.
The targets are for bare-metal environment which is not hosting build tools or a compiler.
The linker used by the targets is the GCC linker from the GCC toolchain cross-compiled for riscv. GNU GPL.
Agreed.
The targets implement libStd almost in its entirety, except for the missing support for process, as this is a bare metal platform. The process
sys\unix
module is currently stubbed to return "not implemented" errors.Target does not (yet) support running tests. We would gladly provide all documentation how to build for the target (where?). It is currently hosted in this README.md file, but will likely be moved to the esp-rs organization. Since the build for the target is driven by cargo and all other tooling is downloaded automatically during the build, there is no need for extensive documentation.
Agreed.
Agreed.
To the best of our knowledge, we believe we are not breaking any other target (be it tier 1, 2 or 3).
To the best of our knowledge, we have not introduced any unconditional use of a feature that affects any other target.
Agreed.