Using dcaf without alloc

[chrysn]

Is your feature request related to a problem? Please describe.

dcaf provides many convenient tools for constrained devices, but requires an allocator.

An allocator is rarely used in OSes such as RIOT OS and Ariel OS, and handling unbounded memory requirements makes it hard to get reliable statements such as "the system can establish and maintain a security association of an administrator while under attack by users who are authorized to establish their connections", which is desirable at least in Ariel.

Describe the solution you'd like

Ideally, dcaf should work with no_alloc – at least in a subset of features.

Describe alternatives you've considered

• Using a per-dcaf-operation allocator (probably a bump allocator) might be possible, but just behind the nightly allocator_api, and would require threading an allocator quite far through about everything.
• The next best alternative I'd come up with is reimplementing parts of dcaf, which is a lot of duplicate work, and unless dcaf would reuse components from there where easily possible, duplicate long-term maintenance.

Additional context

There are some parts where I know this will be hard:

• dcaf builds strongly on ciborium, which itself requires alloc for everything but its low-level parts.

  I think that the parsing / generating side would work well with a no-alloc CBOR library (such as cborium-ll or minicbor), but that'll mean replacing what is currently a Vec<ciborium::Value> or similar (at extension points, where ciborium presents parts that it could not parse) with iterators over parsable CBOR.

• dcaf builds on coset; not sure what can be done there.

• Some COSE structures are not well bounded; for example, creating a serialized Sig_structure in linear memory requires a buffer larger than the original message. (That's also the case with allocation, but there, it can "just allocate" that). Operations therefore either need access to a sufficiently sized buffer, or crypto back-ends that support scatter-gather operations. (Fortunately, not for AEAD operations, where implementers are particularly averse to streaming interfaces: there, ciphertext is already in one place, and only the AAD needs to be composed and streamed).

---

I know this is a large request. If this is something you like to start exploring, I'm happy to contribute. If it is something you don't regard as feasible, are you open to moving parts where it is possible easily (or that are already no-alloc) into a utility crate sharable both by dcaf and tools that operate in no-alloc space?

[pulsastrix]

Hi, and thanks for the feature suggestion.

Using dcaf-rs in no_alloc environments is definitely something we would want to support (at least as a long-term goal), even though it might require a lot of rewriting of existing code.

• Using a per-dcaf-operation allocator (probably a bump allocator) might be possible, but just behind the nightly allocator_api, and would require threading an allocator quite far through about everything.
• The next best alternative I'd come up with is reimplementing parts of dcaf, which is a lot of duplicate work, and unless dcaf would reuse components from there where easily possible, duplicate long-term maintenance.

Personally, I think that the latter option would probably be more elegant in the long term (considering that the original intention is to get away from dynamic allocation), although we should try not to complicate the API for users that do have an allocator available.

Just some of my observations/thoughts regarding this:

• For most of the types currently provided by alloc (e.g., Vec, Box, ...), there is an alternative implementation in the heapless crate, which uses a fixed-size array as the backing data storage.
  • heapless is used in crates such as smoltcp and one of the rust-embedded working group's projects, so it should be maintained well enough to be used as a dependency here.
  • The main trade-off here would be that we'd have to provide reasonable sizes for the backing storage of each field using one of these types.
    • As the backing storage is a fixed size array contained in the respective type, struct fields containing heapless types will always be as large as if they were filled to their max capacity, which must be considered when estimating memory usage, especially if an instance of these structs is stored on the stack.
    • We could also make the sizes configurable during compile time using an approach similar to smoltcp's.
  • We might be able to define an abstraction trait implemented by alloc::vec::Vec and heapless::Vec, which would enable users to choose between the two alternatives.
    • This would come at the cost of adding a generic type parameter to every struct (complicating the API), but by setting a reasonable default users might not even notice this.

• dcaf builds strongly on ciborium, which itself requires alloc for everything but its low-level parts.

  I think that the parsing / generating side would work well with a no-alloc CBOR library (such as cborium-ll or minicbor), but that'll mean replacing what is currently a Vecciborium::Value or similar (at extension points, where ciborium presents parts that it could not parse) with iterators over parsable CBOR.

• I agree, this should be possible, considering that we already have explicit conversion functions from and to ciborium::Value, which would just have to be rewritten to use lower-level encoders/decoders. However, I'm not sure about how to best implement extension points.
  • In order to fully parse a message, the CBOR library we decide on using must support skipping over those extension parts (and ideally returning to their position later on).
  • Providing these unparsed parts (or an iterator over those) to the application requires us to have some (lifetime-restricted) reference, a copy or transferred ownership of the original input bytes. Not sure about the implications of this.

• dcaf builds on coset; not sure what can be done there.

• Some COSE structures are not well bounded; for example, creating a serialized Sig_structure in linear memory requires a buffer larger than the original message. (That's also the case with allocation, but there, it can "just allocate" that). Operations therefore either need access to a sufficiently sized buffer, or crypto back-ends that support scatter-gather operations. (Fortunately, not for AEAD operations, where implementers are particularly averse to streaming interfaces: there, ciphertext is already in one place, and only the AAD needs to be composed and streamed).

• The easiest solution for just getting dcaf-rs usable in no_alloc environments quickly would probably be to disable all COSE-related functionality if no allocator is available. Obviously, this just moves the issue of actually providing a usable COSE library over to the user, but it would at least make it possible to use dcaf-rs without allocation using non-CWT token formats.
  • Decoupling dcaf-rs from the used COSE library might be a desirable goal in itself, as it would allow choosing a more fitting implementation than coset based on the use case, but I don't know what @falko17's opinion is on this.
  • In the long term, we would still have to either rewrite coset to support no_alloc or write an entirely new COSE implementation from scratch that doesn't need allocation (assuming that doing so is feasible at all).
    • We should probably try to transfer all of the current COSE code in src/token/cose, which has nothing to do with ACE-OAuth and is basically just an extension of coset, over to coset first (if its maintainers are open to that).

I know this is a large request. If this is something you like to start exploring, I'm happy to contribute. If it is something you don't regard as feasible, are you open to moving parts where it is possible easily (or that are already no-alloc) into a utility crate sharable both by dcaf and tools that operate in no-alloc space?

We're very much open to contributions if you want to work on this, even though it might take some time for us to review pull requests (as AFAIK both @falko17 and I are currently busy with some other stuff).

[falko17]

I agree with most of @pulsastrix's points. Some short comments on this:

Decoupling dcaf-rs from the used COSE library might be a desirable goal in itself, as it would allow choosing a more fitting implementation than coset based on the use case, but I don't know what @falko17's opinion is on this.

I'm not sure if we'd really want to decouple from the COSE library. At first glance, I think the complexity this brings into dcaf-rs is only worth it if coset cannot be made no_alloc (e.g., if its maintainers disagree), but if a different/new COSE crate could. I think we should communicate with coset's team first to clear this up.

We might be able to define an abstraction trait implemented by alloc::vec::Vec and heapless::Vec, which would enable users to choose between the two alternatives.
This would come at the cost of adding a generic type parameter to every struct (complicating the API), but by setting a reasonable default users might not even notice this.

In principle, I think this is a good idea. Ideally, dcaf-rs should support both environments that have allocators and those that don't, so an abstraction like this (assuming it's possible without complicating its use too much) would be a good fit.

We're very much open to contributions if you want to work on this, even though it might take some time for us to review pull requests (as AFAIK both @falko17 and I are currently busy with some other stuff).

Yes, agreed, contributions are always welcome—it'll probably be some time before either of us can implement a big feature like this ourselves. Starting next year, I should at least have enough time for reviews, though.