Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Embassy HAL implementation for Espressif chips #745

Closed
6 tasks done
MabezDev opened this issue May 2, 2022 · 25 comments
Closed
6 tasks done

Embassy HAL implementation for Espressif chips #745

MabezDev opened this issue May 2, 2022 · 25 comments

Comments

@MabezDev
Copy link
Contributor

MabezDev commented May 2, 2022

We're looking to implement the required traits/executors for Espressif chips, do you have any pointers on what we'll need to implement in our HAL's?

So far it looks like we'll need:

  • Embassy executor for RISCV arch. - Preliminary RISCV support #804
  • Embassy executor for Xtensa arch - Preliminary Xtensa support #805 .
  • Add embedded-hal alpha traits.
  • Add at least one embassy-time::Driver implementation for each chip
  • Implement embedded-hal-async where appropriate.
  • Add an InterruptExecutor for each chip
@lulf
Copy link
Member

lulf commented May 2, 2022

Great to see the interested of running embassy on espressif! 🚀

We're looking to implement the required traits/executors for Espressif chips, do you have any pointers on what we'll need to implement in our HAL's?

So far it looks like we'll need:

* Embassy executor for RISCV arch.

* Embassy executor for Xtensa arch.

👍 ! The 'core' of the executor is arch-independent, but the the different arch-dependent executor parts can be found here:

https://github.com/embassy-rs/embassy/tree/master/embassy/src/executor/arch

The cortex_m one might be the most relevant to look at.

* Add embedded-hal alpha traits.

* Implement embedded-hal-async where appropriate.

In addition to the embedded-hal(-async) traits, embassy has the concept of an opt-in 'timer driver' system timer that is implemented for most HALs, which makes async timers just work. For instance:

https://github.com/embassy-rs/embassy/blob/master/embassy/src/time/driver.rs

and an implementation:
https://github.com/embassy-rs/embassy/blob/master/embassy-nrf/src/time_driver.rs#L122-L126

There is also the critical-section crate that must have implementations for the target architecture (https://crates.io/crates/critical-section). Risc V seems to have some support there already.

There is also some hardware specific stuff in the macros that's needed for users to use the #[embassy::main] macro:

https://github.com/embassy-rs/embassy/blob/master/embassy-macros/src/macros/main.rs#L8-L15

@MabezDev
Copy link
Contributor Author

MabezDev commented May 3, 2022

Thanks for the pointers!

For HAL peripheral implementations I also saw the Unborrow trait used sporadically, I understand what it's doing but I don't know why it's needed? Could you point me in the direction of some docs to explain the requirement of this trait and when to use it?

@Dirbaio
Copy link
Member

Dirbaio commented May 3, 2022

It's to allow creating drivers with &mut singletons. It's very handy when you want to create a driver temporarily, for example for low power. It avoids moving the singletons into the driver and then back out with some .free() method other HALs have, which usually requires Options and unwraps, which is annoying and error-prone.

(It has nothing to do with async, it's a design decision the embassy HALs make, you can do different design decisions if you like)

@osobiehl
Copy link

osobiehl commented May 3, 2022

Hi everyone,
I've been trying to port embassy to the esp32c3 for a couple of weeks now, I have a VERY rough and patchy port working here that awaits a timer and prints hello, world!, among with some other small tests.

I made it by stitching together the assembler code responsible for setting up the interrupt table from [esp-hal] (https://github.com/esp-rs/esp-hal) , as well as using some of the existing traits / implementations from embedded-hal to write to the UART0 peripheral.

Some of what I have working is:

  • thread mode executor using a global flag since riscv doesn't have something similar to cortex_m's asm::sev (thanks @Dirbaio for the tip)
  • time driver using SYSTIMER peripheral, (requires adding 16 MHz to clock frequencies in embassy) allowing allocation of up to 3 alarms (We probably shouldn't use all 3 in case some other peripheral requires a systimer irq, but that's outside my scope). The peripheral uses a 52-bit timer so it should be low-risk in terms of overflowing

Interrupts are handled differently on the ESP32C3 board, I'm not sure if the same interrupt controller is used for other ESP boards, either way, another area that would need porting is interrupts, this would require

  • Removing the blanket implementation for InterruptExt for the Interrupt trait, or at least defining an interrupt_esp32xx.rs without a blanket implementation and using conditional compilation to use that when targeting ESP boards.
  • Finding a nice way to declare CPU interrupts on the ESP32C3 board without having clashes, since there are more interrupt sources than potential interrupts, the approach is not as clear-cut as cortex_m's NVIC, and requires a lot more manual work in terms of enabling and disabling interrupts. I just piggybacked off the interrupt module in esp32-hal and it seems to work reasonably well.

Finally, one last thing to do would be to have compatibility with embassy_macros. I kind of got them working to use the #[riscv-rt::entry] macro for the esp board, but the compiler seems to complain when it comes to cfg attributes in procedural macros. I also don't have any other board than the ESP32C3 so I haven't really been doing compatibility checks.

@MabezDev
Copy link
Contributor Author

MabezDev commented May 4, 2022

This is awesome! Thanks @osobiehl, I'll take a in-depth look soon!

Interrupts are handled differently on the ESP32C3 board, I'm not sure if the same interrupt controller is used for other ESP boards, either way, another area that would need porting is interrupts, this would require

Yes the c3 uses a custom interrupt controller, and the next few chips will too. Longer-term I believe we will integrate the standard PLIC. Perhaps we can find an abstraction that supports PLIC & custom interrupt controllers (I know there are other RISCV chips out there with them) without special behaviour inside embassy.

@MabezDev
Copy link
Contributor Author

I've just opened #804 based on your work @osobiehl, thank you for laying the groundwork! I've also been working on the esp-hal side, including a SystemTimer implementation in esp-rs/esp-hal#76. My current embassy dev branch is here.

@MabezDev MabezDev changed the title Embassy HAL implementation requirements Embassy HAL implementation for Espressif chips Jun 10, 2022
@jessebraham
Copy link
Contributor

jessebraham commented Jun 13, 2022

I've opened a PR in esp-hal implementing a number of the embedded-hal alpha traits: esp-rs/esp-hal#82

Edit: this has now been merged.

@jessebraham
Copy link
Contributor

In case anybody wants to follow progress, I've created some tracking issues in esp-hal regarding the Trait implementations:

embedded-hal alpha traits: esp-rs/esp-hal#69
embedded-hal-async traits: esp-rs/esp-hal#70

@markfarnan
Copy link

How close is this to useable ? Especially on the Xtensa ESP32 base device (not S/C etc)
Failing that, the C3

@jessebraham
Copy link
Contributor

jessebraham commented Jan 2, 2023

@markfarnan as things stand currently:

  • We have have time driver implementations for all currently supported chips (ESP32, ESP32-C2/C3, ESP32-S2/S3) using either TIMG as the timer source or SYSTIMER (with the exception of the ESP32, as is does not have this peripheral). Timer source is selectable via features.
  • We do not presently have async peripheral drivers, however implementing these is high on our priority list and should begin in the coming weeks. Hopefully this will not take too long, I would expect these to be done in Q1, and I will be sure to update the tracking issue linked above as traits are implemented.

@MabezDev
Copy link
Contributor Author

MabezDev commented Feb 9, 2023

A quick update 6~ weeks on:

We have two async drivers in esp-hal:

and we also have two forms of async networking through esp-wifi

@TheButlah
Copy link

TheButlah commented Feb 25, 2023

@Dirbaio embassy switched from using atomic_polyfill in #1125, which breaks riscv. The esp32c3 does not support atomics, and AtomicBool and other atomics don't exist on the platform and do not exist in core.

Some further context from this discussion:

The difference between the riscv32imc-* and the thumbv6m-* rustc compilation targets is that the RISCV one has broken LLVM codegen and that's why it doesn't have the Atomic* API available at all.

In other words, even though embassy doesn't require CAS, rustc doesn't emit even the types themselves.

error[E0432]: unresolved import `core::sync::atomic::AtomicBool`
 --> /home/ryan/.cargo/git/checkouts/embassy-9312dcb0ed774b29/26474ce/embassy-executor/src/arch/riscv32.rs:3:26
  |
3 | use core::sync::atomic::{AtomicBool, Ordering};
  |                          ^^^^^^^^^^ no `AtomicBool` in `sync::atomic`
  |
help: consider importing this struct instead
  |
3 | use core::sync::atomic::{atomic_polyfill::AtomicBool;
  |                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For more information about this error, try `rustc --explain E0432`.
error: could not compile `embassy-executor` due to previous error

Could #1125 be reverted to restore embassy support to esp32c3?

Side note: Could someone more knowledgeable on the subject open an issue in rustc to get the codegen working for this and link to the issue?

@MabezDev
Copy link
Contributor Author

I don't think anything needs to be changed in embassy. We have atomic emulation in esp-hal for the esp32c3 and esp32c2 enabled by default. You just need to add the following rustflags to your project from the riscv-atomic-emulation-trap handler: https://github.com/esp-rs/riscv-atomic-emulation-trap#usage.

Atomic load/stores on RISCV are implemented as a normal load/store (essentially zero cost) and I believe CAS will actually be handled by atomic-polyfill where possible because it is set to polyfill for any riscv target: https://github.com/embassy-rs/atomic-polyfill/blob/80cee72cf0e583e6fccefa8ee463d66ad9aa05d8/build.rs#L31. To ensure that is the case, you could remove the CAS related cfg's from the rustflags list above.

@TheButlah
Copy link

TheButlah commented Feb 26, 2023

doesn't this require trap handlers get taken on any load or store on an atomic? I would assume that is bad for performance.
So far I have been able to avoid the use of imac.

If we don't need CAS, but we do need atomic loads/stores, is using the trap handler really the right approach?

Disclaimer: I have no experience with atomics at the assembly level or codegen. I'm asking about this because I am trying to understand and learn, and am trying to understand what the performance tradeoffs here are.

@Dirbaio
Copy link
Member

Dirbaio commented Feb 26, 2023

#1125 removed atomic_polyfill for uses that were only load/store (no CAS). The esp32c3 hardware (riscv32imc) is capable of doing that.

It is (IMO) a rust bug that load/store atomics aren't exposed in riscv32imc. They are exposed in thumbv6m, which is the same case (can do load/store but no CAS). See rust-lang/rust#99668

That's what should be fixed, instead of adding workarounds to Embassy.

@TheButlah
Copy link

TheButlah commented Mar 1, 2023

Awesome, thanks for the added context :)
I think I misunderstood mabez's message - I thought he meant this would require me to use the trap handler, but I see now that I can just pass the rustflags for Load/Store.

@proegssilb
Copy link

I'm a little confused by current status.

  • Are we still waiting for embedded-hal-async implementations? Or are those entirely complete?
  • Interrupts are still basically not supported at this time, correct?

I'm looking at maybe switching from CircuitPython to Embassy for broader hardware support. In order for the port to make sense, I'd need BLE, i2c, i2s, spi/qspi, and gpio (preferably with interrupts) already working across nrf52840, esp32-s3, and esp32-c6 (as implemented in the Adafruit Feather boards, for the nrf and the s3).

If there's anything I can do to help out, I'm all ears, but I'll be the first to admit limited expertise with low-level hardware work.

@MabezDev
Copy link
Contributor Author

All the eha traits are now implemented (I've updated the check list). The remaining async drivers can be tracked here: esp-rs/esp-hal#361. From your list async i2c,spi,gpio and BLE are already available. Async I2S is being worked on.

@markfarnan
Copy link

markfarnan commented Sep 20, 2023 via email

@MabezDev
Copy link
Contributor Author

Does this make Embassy now useable, (for the peripherals which are done),
or are there other blocking issues ?

Yes, its been more than usable since February this year :)

@proegssilb
Copy link

Thanks for the update, MabezDev. I noticed after the update, interrupts still remain unchecked, and your message didn't mention them. Just for clarity, do you know if interrupts are still a work in progress?

@MabezDev
Copy link
Contributor Author

An interrupt executor != interrupts (See the note about them here: https://embassy.dev/book/dev/runtime.html). Interrupts are essentially a requirements for async, and have been supported in esp-hal since esp-rs/esp-hal#118 & esp-rs/esp-hal#103.

Thanks to @bugadani we do actually have an interrupt executor inside esp-hal, but currently it's only for our Xtensa chips at the moment, which is why I haven't checked it off.

@markfarnan
Copy link

markfarnan commented Sep 20, 2023 via email

@TheButlah
Copy link

TheButlah commented Sep 21, 2023

Check https://github.com/esp-rs/awesome-esp-rust for general esp32 rust stuff, and if you want a more complex firmware that uses it with embassy, you can reference https://github.com/SlimeVR/SlimeVR-Rust/tree/main/firmware.

The best source of information though is the example code in esp-hal: https://github.com/esp-rs/esp-hal/blob/main/esp32c3-hal/examples/embassy_hello_world.rs

@MabezDev
Copy link
Contributor Author

MabezDev commented May 1, 2024

I think we can consider this done, we have excellent embassy support in esp-hal including but not limited to:

  • An Interrupt executor
  • Thread mode, multicore aware, executor
  • Multiple time-driver implementations
  • Many async drivers capable of running on any executor, including embassy-executor
  • async networking with esp-wifi using embassy-net
  • async Bluetooth using the bleps stack, and soon trouble support too.

Thanks to everyone involved in making this possible! ❤️

@MabezDev MabezDev closed this as completed May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants