Device family HALs in rust

[ryankurte]

Hey all,

Has anyone looked at / worked out how to build generic HALs / driver libraries that work across a family of cores instead of writing a peripheral implementation for each? IMO it is an important layer of abstraction to make embedded rust easier both to use and maintain, and I haven't seen it mentioned in any of the roadmaps so far or worked out how to achieve it yet.

As an example of what I am talking about, when using silicon labs cores in C we have the vendor provided emlib that includes <em_device.h>, through some macro wizardry that ends up being replaced with the processor headers, then each of the peripheral driver functions consumes / works on the types exposed in those headers. For an example, see em_gpio.h and em_gpio.c.

So we have one set of drivers to use (and maintain, though in this case by the vendor) that exploit the commonality between device families. This provides a useful abstract interface (ie. SPI mode, frequency rather than registers) and the correct peripherals are either loaded by default where there is one instance (eg. CMU) or injected into the driver where there are multiple (eg. SPI0, SPI1). Also, swapping between MCUs within a family is only a matter of changing the -DEFM32G210F128 argument on the command line and some pin/instance definitions.

The options I have come up with so far to support this in rust are:

• macro-ing the device definition through the HAL package (somehow)
• we could manually implement a set of traits for a given device family, and feed that into svd2rust to match and use them where appropriate.
• we could teach svd2rust to extract common patterns from a set of input svds into traits for use in the HAL.
• we could write the driver layer as a set of macros that assemble the right peripheral registers underneath based on a device config argument.
• we could not worry, build support for single devices only, and duplicate code as required.

The first is imo the most achievable now, but because of the way rust2svd outputs files there is no common trait between a pair of the same peripherals, so I can't see how to abstract across both. The second is IMO the most elegant, in that the crate for a given device could include the family crate and use that to support the SVD output from rust2svd, and applications could then include the family crate and #cfg between device crates, though the HAL is going to need to do the same to handle different functions across devices. I guess that would end up looking something like (from the bottom up):

1. Device definition helpers - #device, #family etc macros
2. Family definition helpers - device and family listing, #efm32device, #efm32family macros
3. HAL implementation - traits for families and devices and peripheral drivers using those to implement the embedded-hal traits discussed in Tooling: Ideal developer experience #6 , Abstract support for peripheral drivers for communication protocols #8 and Define minimal Traits for common embedded peripherals #19
4. Device implementation - rust2svd generated wrappers using HAL traits
5. Application / drivers on top of this and abstract (except for configuration) from both the device and processor family.

However it's achieved it seems like there are going to be some interdependencies, but I would appreciate any feedback / thoughts / ideas on how to best solve it ^_^

[jcsoo]

I've been working on this exact problem for about a year now, and you bring up a lot of great points about what's desirable and how hard it will be to accomplish. I believe the only workable approach is to have layers of crates covering architecture-common, vendor-common, subfamily-common, and eventually chip-specific features. There are some use cases where developers will want to target purely high-level traits, but it is just as important to be able to work with highly optimized and tested vendor-specific peripheral drivers that expose all the functionality available.

Unfortunately, I don't think SVD is suitable except as a starting point. It simply wasn't designed for that purpose and doesn't support the necessary concepts, not to mention that the quality and style of SVD files varies widely between vendors when they are even available. My view is that vendors see SVD primarily as a convenient output format to use in their GUI device-aware debuggers, not as a source of truth to be used for source generation.

I'm also wary about a macro-heavy approach to configuration and peripheral definition. Those are areas where debuggability and understandability are really important, especially for embedded device programmers coming to rust, and complex macro systems can be opaque even for experienced Rust programmers.

I think in most cases where someone is targeting an application at multiple boards / MCU models (i.e. not just variants with different amounts of memory), it will make sense to break the higher level application logic into a separate crate and have individual top-level crates that do the board / MCU-specific configuration. So you have something like:

Top Level Crate (initialization and configuration)
Application Crate (application logic)
Device-independent Library Crates
Peripheral Crates (Board + MCU independent peripherals such as sensors, displays and radios)
Generic Embedded Trait Crates
Board Crate
MCU Model Crate
MCU Family Common Crate
Architecture Common Crate

depending on the complexity of your application and how broadly you expect it to be deployed.

[japaric]

Has anyone looked at / worked out how to build generic HALs / driver libraries
that work across a family of cores instead of writing a peripheral
implementation for each?

I have done some work in this area. Here's a WIP HAL as a set of traits. You
can find an implementation of those traits here; even though the crate says
"blue-pill" -- that's the name of the board I'm using -- the implementation
works for different devices in the family of STM32F103xx devices.
Finally here are some applications that make use of those crates.

The applications don't directly use registers but the slightly higher level HAL
that hides the details about those. You'll still see references to peripherals
(e.g. TIM2) in the application code; those are there to (a) eliminate data races
and (b) help you reason about race conditions in the present in interrupts. I
still haven't figured out a good way to "erase" the peripheral names without
throwing performance out of the window and putting all the burden of reasoning
about race conditions on the user.

macro-ing the device definition through the HAL package (somehow)

Rust doesn't have macros in the C sense (a "preprocessor"). By "macro-ing the
device definition" do you mean conditional compilation using #[cfg]
attributes?

There have been some recent discussion about using #[cfg] in device crates,
the stuff that svd2rust generates, in rust-embedded/svd2rust#122 more as a
configuration mechanism rather than for code reuse though. But there I note the
problems I see with #[cfg]s: mainly their virality -- once you use them in the
lower layers they "percolate" through all the higher layers -- and the issues
about how to activate them using Cargo features. Using #[cfg] for code reuse
would probably run into similar issues.

we could manually implement a set of traits for a given device family, and
feed that into svd2rust to match and use them where appropriate.

What set of traits do you have in mind? Depending on how you define them it
could be rather hard to make svd2rust auto derive them.

Something like this has been brought up in the svd2rust repo before in
rust-embedded/svd2rust#96. IMO it's better to reduce the number of types rather than
create traits and have a bunch of very similar implementations of them. With an
example: if the types (structs) stm32f103xx::TIM2 and stm32f20x::TIM2 are
exactly the same then why do you even have two types to begin with? There
should only be one type, e.g. stm32_common::TIM2, and that should be simply
re-exported in the stm32f103xx and stm32f20x crates. If you go down the
trait route you'll end with a Tim2 trait with duplicate implementations: impl Tim2 for stm32f103xx::TIM2 and impl Tim2 for stm32f20x::TIM2; the
implementations would be the exactly same so this would still be considered code
duplication.

we could teach svd2rust to extract common patterns from a set of input svds
into traits for use in the HAL.

This sounds like rust-embedded/svd2rust#96. I still prefer less types and re-exports
rather than traits. Generic programming is not precisely pleasant, specially
when a bunch of bounds are involved; I'd prefer to reduce the number of generic
code that has to be written and read.

because of the way rust2svd outputs files there is no common trait between a
pair of the same peripherals

There's already a common trait between several instances of the same peripheral.
svd2rust generates something like this:

// "newtypes"
struct TIM2(tim2::RegisterBlock);
struct TIM3(tim2::RegisterBlock);
struct TIM4(tim2::RegisterBlock);

// the common type: a general purpose timer
mod tim2 {
    struct RegisterBlock {
        pub CR1: CR1,
        // ...
    }
}

// the common trait
impl Deref for TIM2 { type Target = tim2::RegisterBlock; /* .. */ }
impl Deref for TIM3 { type Target = tim2::RegisterBlock; /* .. */ }
impl Deref for TIM4 { type Target = tim2::RegisterBlock; /* .. */ }

// instances
const TIM2: Peripheral<TIM2> = ..;
const TIM3: Peripheral<TIM3> = ..;
const TIM4: Peripheral<TIM4> = ..;

This lets you write generic code that works with all the instances of a general
purpose timer:

// generic function
fn foo<T>(tim: &T) where T: Deref<tim2::RegisterBlock> {
    let tim2: &tim2::RegisterBlock = tim.deref();

    tim2.arr.write(..);
}

// that works with all the instances
foo(&TIM2);
foo(&TIM3);
foo(&TIM4);

Device definition helpers - #device, #family etc macros

What are #device and #family macros? Do you mean #[cfg(device = "foo")]?
"compile this item if the device is 'foo'?".

[ryankurte]

Hey, thanks for the responses! Pretty much everywhere I put "macro" I should have put conditional compilation (#[cfg]), language context switching troubles :-/

The top level embedded-hal is an awesome concept for the top level (ie. how to implement/use generic peripherals) abstraction (though I am sure likely to lead to some contention ^_^).

The hal implementation in blue pill is also great, but still depends on explicit definition of the f103xx.

Totally agree about the elegance of sharing concrete types across families (ie stm32_common) and re-exporting them, I think that is what was missing from my world view and it makes much more sense than traits. And with those in common it's possible to write a generic stm32_hal that uses types from stm32_common that are injected from the actual implementation (say, stm32f429_hal) in the application.

I would argue it is not necessary to erase peripheral names, having them injected into/through the hal(s) from the application level makes it clear what is in use underneath, and allows for board support packages or platform definitions that define useful names for peripherals where required.

And within the stm32_hal you an use device/family conditional compilation (#[cfg(device = "foo")], #[cfg(family = "bar")] to handle any nasty errata if (or when, it's embedded 😕) required.

I didn't realise rust2svd did common traits, on further investigation it looks like the vendor SVD files I am playing with don't use derivedFrom and just redeclare every peripheral 🙁. Fortunately any mechanism to support common types / rust2svd#96 could totally solve this problem too.

So, what is required to achieve it, and are there any useful small tasks we can undertake to help get there? It looks like a formalisation of the abstraction layers like @jcsoo's above and support for some kind of shared types (SVD / Rust objects / Something else) in svd2rust would be the first blockers?

Also, thanks for all the great work you do! Very excited for this future of embedded rust.

[japaric]

support for some kind of shared types (SVD / Rust objects / Something else) in svd2rust

left some tentative next steps for that issue in rust-embedded/svd2rust#96 (comment)

[jamesmunns]

I feel like this issue should probably be revisited. CC'ing the people I know who are particularly vocal about sharing code within a family:

@ryankurte
@therealprof
@japaric
@hannobraun
@adamgreig

(No priority on this, but I am currently triaging issues.

[GregWoods]

some of my comments in #62 (about a higher level abstraction to the hal) apply to this issue. I show how the code for the stm32f1 and stm21f4 differ when setting up a pin.

[jamesmunns]

Closing this as part of 2024 triage.

Today, this is typically the scope of individual HAL implementors, I'd suggest looking at what embassy-rs has done with the STM32 metapac, and code generation for abstracting over shared peripherals.

If you think this was closed incorrectly, please leave a comment and we can revisit this decision in the next weekly WG meeting on Matrix.