Action message support

[nwn]

This implements action support in rosidl_generator_rs and rosidl_runtime_rs. It's a selective rebase of the original changes made in #295 and #410, applying only changes to the two rosidl packages, without any modification to rclrs. The intention here is to merge these changes sooner so that we can split the message support packages into a separate repo and add them to the ROS buildfarm.

The general approach follows the same pattern as the existing message and service generation. However, the additional message wrapping and introspection done on actions means that we need generic access to certain message internals. In rclcpp, this is done using duck typing on the Action and Action::Impl template parameters. However, Rust is stricter about this and requires trait bounds to ensure that generic type parameters can be accessed in a given way. As a result, a series of trait methods are defined in ActionImpl that enable a client library like rclrs to generically create and access the various message types.

Certain of these methods involve accessing timestamps and UUIDs in messages. To avoid adding a dependency in rosidl_runtime_rs on builtin_interfaces and unique_identifier_msgs, these are represented in a more primitive form as (i32, u32) and [u8; 16] in the signatures, respectively. This is rather ugly, so ideas are welcome.

The existing srv.rs.em file is split into separate "idiomatic" and "RMW" implementations, similarly to how messages are already handled. This makes embedding the service template into the action.rs.em easier.

[Guelakais]

I think it would be a good idea to create Rust packages on a test basis and check whether they generate warnings. I would also like to see the generated rust structures include a few derivable traits. I have attached a list. It all looks great in itself.

Common Derivable Rust Traits

In Rust, the derive attribute allows for automatic implementation of certain traits for structs and enums, simplifying the process of writing boilerplate code. Here are some of the most common derivable traits:

1. Clone: This trait allows for creating a duplicate of a value. When you derive Clone, it enables the use of the clone method to create a copy of the instance.

2. Copy: This trait provides "copy semantics" instead of "move semantics." Types that implement Copy can be duplicated simply by copying bits, which is useful for simple data types.

3. Debug: This trait enables formatting a value using the {:?} formatter, which is particularly useful for debugging purposes.

4. Default: The Default trait allows you to create a default value for a type. When derived, it implements the default function, which calls the default function on each field of the type.

5. PartialEq: This trait allows for equality comparisons between instances of a type. Deriving PartialEq enables the use of the == and != operators.

6. Eq: This trait is a marker trait that indicates that a type's equality comparison is reflexive, symmetric, and transitive. It is typically derived alongside PartialEq.

7. PartialOrd: This trait allows for partial ordering of instances, enabling the use of comparison operators like <, >, <=, and >=.

8. Ord: This trait provides a total ordering for instances of a type. It is derived from PartialOrd and allows for the use of the sort method.

9. Hash: This trait allows for computing a hash from an instance, which is essential for types that will be used in hash-based collections like HashMap.

[jhdcs]

I think one of the difficulties of deriving traits for message types is that we have very little information about the individual messages before compile time. We cannot be sure that any of these traits are derivable before the generators create the code.

Additionally, these messages are interacted with through the FFI boundary, which can potentially cause some weirdness at the best of times. I would be cautious about the automatic derivation of any traits unless we are 100% certain that we aren't going to have a problem introduced due to the FFI interactions. Admittedly this is a rare issue, but I feel it's best to be cautious when dealing with FFI.

Is there a specific reason that you would like to see these traits automatically derived?

[mxgrey]

I think one of the difficulties of deriving traits for message types is that we have very little information about the individual messages before compile time. We cannot be sure that any of these traits are derivable before the generators create the code.

I'm not so sure this is true. We know three important things:

• Every message definition is a struct composed of some combination of other messages or of ROS message primitives (e.g. f32, f64, u32, i32, [T], ...).
• Every ROS message primitive can implement Clone, Debug, Default, PartialEq, Eq, PartialOrd, Ord, and Hash (the only one missing is Copy).
• Every struct composed of things that implement those traits can also derive those traits

So I believe we could trivially derive all the popular traits besides Copy automatically for all message definitions.

[mxgrey]

Is there a specific reason that you would like to see these traits automatically derived?

I'm strongly in favor of providing the implementations for the sake of developer quality of life.

For better or worse, ROS message data structures play a big role in the internal mechanics of most ROS nodes.

• Being able to easily clone when needed can be vital to writing reasonable control flows.
• Debug would help people inspect their nodes and write useful log messages
• Hash and Eq help with caching and recognizing redundant messages
• PartialOrd and Ord can help with organizing or sorting data, but these are probably the least important of the traits, so I wouldn't feel strongly about including this or not.

[mxgrey]

All that said, I might suggest we save traits for a follow up PR.

[mxgrey]

Certain of these methods involve accessing timestamps and UUIDs in messages. To avoid adding a dependency in rosidl_runtime_rs on builtin_interfaces and unique_identifier_msgs, these are represented in a more primitive form as (i32, u32) and [u8; 16] in the signatures, respectively. This is rather ugly, so ideas are welcome.

It would definitely be nice if we can clean up that situation a bit. I don't love what I'm about to suggest, but here's a stab at having something a bit more pleasant:

• Define newtype structs in the rosidl_runtime_rs such as Timestamp and Uuid to wrap this data
• Add a special condition to rosidl_generator_rs to check if we are generating bindings for builtin_interfaces::Time or unique_identifier_msgs::Uuid
• If we are generating one of the special cases, then include an impl From<Newtype> for Message for each. Perhaps also give each message struct a method to convert it into the relevant rosidl_runtime_rs newtype.

Let me know if anyone has thoughts on that. I can take a stab at drafting it if there are no strong objections.

[Guelakais]

I think one of the difficulties of deriving traits for message types is that we have very little information about the individual messages before compile time. We cannot be sure that any of these traits are derivable before the generators create the code.

Additionally, these messages are interacted with through the FFI boundary, which can potentially cause some weirdness at the best of times. I would be cautious about the automatic derivation of any traits unless we are 100% certain that we aren't going to have a problem introduced due to the FFI interactions. Admittedly this is a rare issue, but I feel it's best to be cautious when dealing with FFI.

Is there a specific reason that you would like to see these traits automatically derived?

In my opinion, this makes handling the structs feel more idiomatic. When I program other things with rust, I expect these standard traits from the imported structs. It also makes the handling of the structs simpler, as certain things like comparisons or the output of the structs in the standard output then already work. You get a better feel for what the struct is doing. It's also a bit awkward every time you have to implement it yourself. In many cases it works with the newtype pattern. In some cases you have to dig deeper into the material. It doesn't matter if it doesn't work. I thought it would be a nice idea.
On the other hand: Ros messages are defined by design so that they can only consist of primitive data types. This goes so far that it is basically not possible to transmit enum symbols with Ros. As far as I know, strings in Ros are equivalent to rusts String.

[nwn]

@Guelakais I assume we're only talking about the message types, since the top-level action type is an empty struct that's only useful as a generic type parameter. It's not useful to create an actual instance of this type, so I don't see a need for any trait derivations there.

For the message types, I agree that it's useful to have certain common traits implemented on them. Regarding the list you gave:

• Clone, Debug, PartialEq, and PartialOrd are already derived for today's message types.
• Default is manually implemented for message types, using the *__init() functions generated by rosidl_generator_c.
• Copy is not universally possible since some messages may contain non-Copy fields like strings and sequences. Since we can't always determine whether a non-primitive field is Copy or not, we would only be able to implement this for messages containing only primitive Copy fields.
• Eq, Ord, and Hash are not universally applicable since the primitives f32 and f64 don't implement these traits. This could be worked around using something like ordered-float to wrap the fields in the implementations, but I'm not sure if that is always correct.

So the traits that are trivially derivable are already done.

In any case, I'd say this should be discussed in a separate issue since the message generation isn't actually touched in this PR.

[nwn]

@mxgrey

Certain of these methods involve accessing timestamps and UUIDs in messages. To avoid adding a dependency in rosidl_runtime_rs on builtin_interfaces and unique_identifier_msgs, these are represented in a more primitive form as (i32, u32) and [u8; 16] in the signatures, respectively. This is rather ugly, so ideas are welcome.

It would definitely be nice if we can clean up that situation a bit. I don't love what I'm about to suggest, but here's a stab at having something a bit more pleasant:

* Define newtype structs in the `rosidl_runtime_rs` such as `Timestamp` and `Uuid` to wrap this data

* Add a special condition to `rosidl_generator_rs` to check if we are generating bindings for `builtin_interfaces::Time` or `unique_identifier_msgs::Uuid`

* If we are generating one of the special cases, then include an `impl From<Newtype> for Message` for each. Perhaps also give each message struct a method to convert it into the relevant `rosidl_runtime_rs` newtype.

Let me know if anyone has thoughts on that. I can take a stab at drafting it if there are no strong objections.

An alternative we could consider would be this:

• Redefine ActionImpl as rosidl_runtime_rs::ActionImpl<TimeMsg, UuidMsg>, taking two generic type parameters that it uses in all the relevant spots in its method signatures.
• In rosidl_generator_rs, where the generated code already depends on the concrete builtin_interfaces and unique_identifier_msgs packages, have the action struct impl rosidl_runtime_rs::ActionImpl<builtin_interfaces::msg::rmw::Time, unique_identifier_msgs::msg::rmw::UUID>. The method implementations would then use the concrete message types.
• In rclrs, use rosidl_runtime_rs::ActionImpl<builtin_interfaces::msg::rmw::Time, unique_identifier_msgs::msg::rmw::UUID> as a trait bound on actions so that we can access the concrete methods.

Without trait aliases, this might be a bit cumbersome to work with on the rclrs side, but it avoids special casing in the generator.

[Guelakais]

@Guelakais I assume we're only talking about the message types, since the top-level action type is an empty struct that's only useful as a generic type parameter. It's not useful to create an actual instance of this type, so I don't see a need for any trait derivations there.

For the message types, I agree that it's useful to have certain common traits implemented on them. Regarding the list you gave:

* `Clone`, `Debug`, `PartialEq`, and `PartialOrd` are already derived for today's message types.

* `Default` is manually implemented for message types, using the `*__init()` functions generated by `rosidl_generator_c`.

* `Copy` is not universally possible since some messages may contain non-`Copy` fields like strings and sequences. Since we can't always determine whether a non-primitive field is `Copy` or not, we would only be able to implement this for messages containing only primitive `Copy` fields.

* `Eq`, `Ord`, and `Hash` are not universally applicable since the primitives `f32` and `f64` don't implement these traits. This could be worked around using something like [ordered-float](https://docs.rs/ordered-float/latest/ordered_float/struct.OrderedFloat.html) to wrap the fields in the implementations, but I'm not sure if that is always correct.

So the traits that are trivially derivable are already done.

In any case, I'd say this should be discussed in a separate issue since the message generation isn't actually touched in this PR.

OH, I didn't know that. If I didn't know that - I should make a pull request to improve the documentation of the codegenerator. :)

I didn't want to cause so much confusion. all in all, the integration of actions is great.

A small note: The communication system of ros2 has strong dependencies on the DDS interface. Depending on which DDS is used here, this can lead to strange effects. How does the rust api handle this?

[mxgrey]

In rclrs, use rosidl_runtime_rs::ActionImpl<builtin_interfaces::msg::rmw::Time, unique_identifier_msgs::msg::rmw::UUID> as a trait bound on actions so that we can access the concrete methods.

This is a very interesting idea for how to break the circular dependency. One nitpick I might have is that if a trait has generic parameters, that implies that users should expect it to be possible for the same data structure to implement the same trait with different generic arguments. I think a slightly better arrangement might be to use associated types:

pub trait ActionImpl {
    type GoalUuid;
    type Timestamp;
    ...
}

then if rosidl_generator_rs can just always use

impl ActionTrait for <T> {
    type GoalUuid = unique_identifier_msgs::msg::rmw::UUID;
    type Timestamp = builtin_interfaces::msg::rmw::Time;
}

Then rclrs can use

Action: ActionImpl<GoalUuid = unique_identifier_msgs::msg::rmw::UUID, Timestamp = builtin_interfaces::msg::rmw::Time>

as a trait bound when needed.

That being said, I don't see anything wrong with special-casing builtin_interfaces and unique_identifier_msgs since those are foundational message packages that play a special role in the internal mechanics of actions. I suspect that having these two special cases in the message generation would be a small price to pay to avoid juggling lots of generics that aren't otherwise necessary.

[mxgrey]

Thanks for splitting this out as its own PR. I wound up spending most of my reviewing time by looking at #410 anyway because it's difficult to understand the traits without the context of how they will be used, but it's nice that we can move along the message generation and enable rosidl_generator_rs to get onto the build farm.

I just left one inline comment about a fix that's needed, and I opened this PR to make a style refactoring recommendation. Besides those items, I think this is good to go.

On the topic of UUID and timestamps, after looking over the actual usage I think the current approach is fine, and it's not worth bringing in generic parameters or associated types. But I will point out that rclrs::GoalUuid could be moved into rosidl_runtime_rs and then publicly re-exported in rclrs. Then it could be used in the API of ActionImpl. I don't think this is important either way (the current approach is perfectly fine; end users won't touch that interface anyway), but I thought I'd point out that it's another option to consider.

[mxgrey]

This PR looks good to me. I believe this is ready for a maintainer review.

[maspe36]

Thank you for picking up this work @nwn! We are closer than we've ever been to having action support thanks to your contributions.

---

I know this may be asking a lot for such complex message generation, but can you add some high level docs? It doesn't need to be in depth, that's a completely different task to fully document our message generation pipeline.

But, something that would help a new developer approach this code could go a long way.

[maspe36]

This is a little confusing to me. Looking at (what I believe to be) the impl for this typesupport function (link)

1. Why are we actually returning a void pointer here? This function is not marked as unsafe, but any handling of the return value may well need to use unsafe code. Could we not return a well formed type, such as rosidl_service_type_support_t?

2. This function seems to initialize the request and response members, but I don't actually see where those members are. The @(type_name) struct appears empty. How is this working?

[nwn]

1. You're right, we could have this return a more specific type than just c_void. However, doing so hasn't been necessary yet, and this is following the same pattern as the existing get_type_support functions for messages and services. To make this happen, we would have to create a Rust binding for the rosidl_{message,service,action}_type_support_t structs, probably in rosidl_runtime_rs. I would be inclined to leave this for a separate PR to avoid growing the scope of this one.
2. I'm not sure what you mean here. The @(type_name) struct is empty since it's not really meant to be instantiated. It would only be used as a generic impl Action argument and to access the specific Goal, Feedback, and Result associated message types.

[maspe36]

1. Sure, I'm fine leaving this to another PR. Just took a look at our usages (1) of this function historically and we do have as casts everywhere. as casts can be problematic. So it would be nice to eventually not need this. However, maybe there is some other reason I'm missing as to why we return void* here.

2. I was referring to the data that function actually mutates in its implementation. I was misunderstanding and assumed that we actually held a handle to that data, but apparently we do not. It is a global variable managed by I guess the rosidl_runtime(?) that will be generated for each service.

static rosidl_service_type_support_t @(function_prefix)__@(spec.srv_name)_service_type_support_handle = {
  0,
  &@(function_prefix)__@(spec.srv_name)_service_members,
  get_service_typesupport_handle_function,
};

This is outside the scope of this PR though. Just calling attention to it because we've been bitten by global variables we directly interact with via FFI before, see #386

[maspe36]

I thought we didn't want to have a dependency on unique_identifier_msgs here

[nwn]

We want to avoid creating a dependency on unique_identifier_msgs from the rosidl_runtime_rs package, since the latter is a dependency of all message packages. This would cause a cyclic dependency.

However, the crates generated by the rosidl_generator_rs do have dependencies on any message packages they use as fields. In this case, unique_identifier_msgs/msg/UUID is a field of the underlying goal service request type.

[maspe36]

Ahh, I see. So the function signature of create_goal_request can't reference unique_identifier_msgs but the impl of that function actually can.

Yeah, we should explore some of the ideas discussed earlier to simplify this. Not needed for this PR though.

[maspe36]

nit: This can be done in another PR because this is not the only place in ros2_rust 😅 but this std::os::raw::c_void type is an alias to std::ffi::c_void which is the more appropriate type to use.

std::os::raw::c_void only exists to maintain backwards compatibility with the Rust compiler <1.30 (until 1.1). Here is the RFC where this was introduced, RFC 2521.

[nwn]

Sure, I've updated all the uses in rosidl_runtime_rs and rosidl_generator_rs to use std::ffi::c_void instead. I'll leave rclrs as-is for this PR.

[nwn]

I know this may be asking a lot for such complex message generation, but can you add some high level docs?

@maspe36 I've added a short doc page under docs/message-generation.md. Let me know what you think.

[maspe36]

The high level doc looks great! Thanks for all your hard work @nwn

[maspe36]

Ahh, I see. So the function signature of create_goal_request can't reference unique_identifier_msgs but the impl of that function actually can.

Yeah, we should explore some of the ideas discussed earlier to simplify this. Not needed for this PR though.