From d6a298ea0d4b59a854c6a201c8df40ad36d348b9 Mon Sep 17 00:00:00 2001 From: methylDragon Date: Tue, 14 Jun 2022 17:50:38 -0700 Subject: [PATCH 1/5] Add nested type description section Signed-off-by: methylDragon --- rep-2011.rst | 117 ++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 110 insertions(+), 7 deletions(-) diff --git a/rep-2011.rst b/rep-2011.rst index 581dedfd4..b9e5293af 100644 --- a/rep-2011.rst +++ b/rep-2011.rst @@ -273,7 +273,7 @@ The final form of these interfaces should be found in the reference implementati string failure_reason # Empty if 'successful' was true, otherwise contains details on why it failed string type_description_raw # The idl or msg file, with comments and whitespace - TypeDescription type_description # The parse type description which can be used programmatically + TypeDescription type_description # The parsed type description which can be used programmatically string serialization_library string serialization_version @@ -290,17 +290,72 @@ And the ``IndividualTypeDescription`` type: .. code:: - FIELD_TYPE_INT = 0 - FIELD_TYPE_DOUBLE = 1 - # ... and so on - - uint8_t[] field_types + string type_name + string[] field_types string[] field_names These naive examples of the interfaces just give an idea of the structure but perhaps do not yet consider some other complications like field annotations and more advanced IDL (generically all "interface description languages" not just the OMG-IDL that DDS uses) have. -.. TODO:: Should we use strings instead of integer enum for the ``field_types``? +.. TODO:: Should we use strings instead of integer enum for the ``field_types``? (methylDragon: We should use strings to support custom/nested types.) + +Nested TypeDescription Example +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The ``TypeDescription`` message type shown above also supports the complete description of a type that contains other types (a nested type), up to an arbitrary level of nesting. +Consider the following example: + +.. code:: + + # A.msg + B b + C c + # B.msg + bool b_bool + + # C.msg + D d + + # D.msg + bool d_bool + +The corresponding ``TypeDescription`` for ``A.msg`` will be as follows: + +.. code:: + + # A: TypeDescription + type_description: A_IndividualTypeDescription + referenced_type_descriptions: [B_IndividualTypeDescription, + C_IndividualTypeDescription, + D_IndividualTypeDescription] + +With the referenced type descriptions accessible as ``IndividualTypeDescription`` types in the ``referenced_type_descriptions`` field of ``A`` (and also for `A`, its own ``type_description`` field. +In the case where a type description contains no referenced types (i.e., when it has no fields, or all of its fields are primitive types), the ``referenced_type_descriptions`` array will be empty. + +.. code:: + + # A: IndividualTypeDescription + type_name: "A" + field_types: ["B", "C"] + field_names: ["b", "c"] + + # B: IndividualTypeDescription + type_name: "B" + field_types: ["bool"] + field_names: ["b_bool"] + + # C: IndividualTypeDescription + type_name: "C" + field_types: ["D"] + field_names: ["d"] + + # D: IndividualTypeDescription + type_name: "D" + field_types: ["bool"] + field_names: ["d_bool"] + +In order to handle the type of a nested type such as ``A``, the receiver can use the ``referenced_type_descriptions`` array as a lookup table keyed by ``type_name`` to obtain the type information of a referenced type. +This type handling process can also support any recursive level of nesting (e.g. while handling A, C is encountered as a nested type, C can then be looked up using the top level ``referenced_type_descriptions`` array). Additional Notes for TypeDescription Message Type ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -578,6 +633,54 @@ Additionally, the option to add a configuration option to choose what contents t As for the format of the type description, using the ROS interfaces to describe the type, as opposed to an alternative format like XML, JSON, or something like the TypeObject defined by DDS-XTypes, makes it easier to embed in the ROS Service response. It also prevents unnecessary coupling with third-party specifications that could be subject to change and reduces the formats that need to be considered on the receiving end of the ROS Service call. +TypeDescription Structure +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Representing Field Type as Strings +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The choice to use a ``string`` array as opposed to a ``uint8`` array to represent the field types of a description is to support custom user-defined type names. + +If, however, space becomes a concern (e.g. when discovery causes network bandwidth usage to spike), it would be advisable to instead define the ``TypeDescription`` type as such: + +.. code:: + + string[] field_definitions + uint8_t[] field_types + string[] field_names + +Where ``field_types`` stores indices to ``field_definitions``, such that the message: + +.. code:: + + # example.msg + uint8 a + B b + C c + uint8 d + +Results in the following ``TypeDescription``: + +.. code:: + + field_definitions: ["uint8", "B", "C"] + field_types: [0, 1, 2, 0] + field_names: ["a", "b", "c", "d"] + + +Using an Array to Store Referenced Types +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Some alternatives to using an array of type descriptions to store referenced types in a nested type were considered, including: + +- Storing the referenced types inside the individual type descriptions and accessing them by traversing the type description tree recursively instead of using a lookup table. + + - Rejected because the IDL spec does not allow for a type description to store itself, and also because it could possibly introduce duplicate, redundant type descriptions in the tree, using up unnecessary space. + +- Sending referenced types in a separate service call or message. + + - Rejected because needing to collate all of the referenced types on the receiver end introduces additional implementation complexity, and also increases network bandwidth with all the separate calls that must be made. + Backwards Compatibility ======================= From f6eb33bc74d81740a35aa18ff355072564edbd39 Mon Sep 17 00:00:00 2001 From: methylDragon Date: Tue, 21 Jun 2022 14:16:23 -0700 Subject: [PATCH 2/5] Use Field type in message descriptions Signed-off-by: methylDragon --- rep-2011.rst | 100 ++++++++++++++++++++++++++++++++------------------- 1 file changed, 64 insertions(+), 36 deletions(-) diff --git a/rep-2011.rst b/rep-2011.rst index b9e5293af..cc0ccc525 100644 --- a/rep-2011.rst +++ b/rep-2011.rst @@ -291,12 +291,22 @@ And the ``IndividualTypeDescription`` type: .. code:: string type_name - string[] field_types - string[] field_names + Field[] fields -These naive examples of the interfaces just give an idea of the structure but perhaps do not yet consider some other complications like field annotations and more advanced IDL (generically all "interface description languages" not just the OMG-IDL that DDS uses) have. +And the ``Field`` type: + +.. code:: -.. TODO:: Should we use strings instead of integer enum for the ``field_types``? (methylDragon: We should use strings to support custom/nested types.) + NESTED_TYPE = 0 + FIELD_TYPE_INT = 1 + FIELD_TYPE_DOUBLE = 2 + # ... and so on + + uint8_t field_type + string field_name + string nested_type_name # If applicable (when field_type is 0) + +These naive examples of the interfaces just give an idea of the structure but perhaps do not yet consider some other complications like field annotations and more advanced IDL (generically all "interface description languages" not just the OMG-IDL that DDS uses) have. Nested TypeDescription Example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -336,25 +346,50 @@ In the case where a type description contains no referenced types (i.e., when it # A: IndividualTypeDescription type_name: "A" - field_types: ["B", "C"] - field_names: ["b", "c"] + fields: [A_b_Field, A_c_Field] # B: IndividualTypeDescription type_name: "B" - field_types: ["bool"] - field_names: ["b_bool"] + fields: [B_b_bool_Field] # C: IndividualTypeDescription type_name: "C" - field_types: ["D"] - field_names: ["d"] + fields: [C_d_Field] # D: IndividualTypeDescription type_name: "D" - field_types: ["bool"] - field_names: ["d_bool"] + fields: [D_d_bool_Field] + +With the corresponding ``Field`` fields: + +.. code:: + + # A_b_Field + field_type: 0 + field_name: "b" + nested_type_name: "B" + + # A_c_Field + field_type: 0 + field_name: "c" + nested_type_name: "C" -In order to handle the type of a nested type such as ``A``, the receiver can use the ``referenced_type_descriptions`` array as a lookup table keyed by ``type_name`` to obtain the type information of a referenced type. + # B_b_bool_Field + field_type: 9 # Suppose 9 corresponds to a boolean field + field_name: "b_bool" + nested_type_name: "" # Empty if primitive type + + # C_d_Field + field_type: 0 + field_name: "d" + nested_type_name: "D" + + # D_d_bool_Field + field_type: 9 + field_name: "d" + nested_type_name: "" + +In order to handle the type of a nested type such as ``A``, the receiver can use the ``referenced_type_descriptions`` array as a lookup table keyed by the value of ``Field.nested_type_name`` or ``IndividualTypeDescription.type_name`` (which will be identical for a given type) to obtain the type information of a referenced type. This type handling process can also support any recursive level of nesting (e.g. while handling A, C is encountered as a nested type, C can then be looked up using the top level ``referenced_type_descriptions`` array). Additional Notes for TypeDescription Message Type @@ -636,37 +671,30 @@ It also prevents unnecessary coupling with third-party specifications that could TypeDescription Structure ~~~~~~~~~~~~~~~~~~~~~~~~~ -Representing Field Type as Strings -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Representing Fields as An Array of Field Types +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The choice to use a ``string`` array as opposed to a ``uint8`` array to represent the field types of a description is to support custom user-defined type names. - -If, however, space becomes a concern (e.g. when discovery causes network bandwidth usage to spike), it would be advisable to instead define the ``TypeDescription`` type as such: +The use of an array of ``Field`` messages was balanced against using two arrays in the ``IndividualTypeDescription`` type to describe the field types and field names instead, e.g.: .. code:: - string[] field_definitions - uint8_t[] field_types - string[] field_names - -Where ``field_types`` stores indices to ``field_definitions``, such that the message: + # Rejected IndividualTypeDescription Variants -.. code:: + # String variant + string type_name + string field_types[] + string field_names[] - # example.msg - uint8 a - B b - C c - uint8 d - -Results in the following ``TypeDescription``: - -.. code:: + # uint8_t Variant + string type_name + uint8_t field_types[] + string field_names[] - field_definitions: ["uint8", "B", "C"] - field_types: [0, 1, 2, 0] - field_names: ["a", "b", "c", "d"] +The string variant was rejected because using strings to represent primitive types wastes space, and will lead to increased bandwidth usage during the discovery and type distribution process. +The uint8_t variant was rejected because uint8_t enums are insufficiently expressive to support nested message types. +The use of the ``Field`` type, with a ``nested_type_name`` field that defaults to an empty string mitigates the space issue while allowing for support of nested message types. +Furthermore, it allows the fields to be described in a single array, which is easier to iterate through and also reduces the chances of any errors from mismatching the array lengths. Using an Array to Store Referenced Types ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ From 6c1e27132dbf2c2b0be24c4c5b9bed266056d8db Mon Sep 17 00:00:00 2001 From: methylDragon Date: Thu, 7 Jul 2022 17:52:57 -0700 Subject: [PATCH 3/5] Fix unmatched parenthesis Signed-off-by: methylDragon --- rep-2011.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rep-2011.rst b/rep-2011.rst index cc0ccc525..eb9e6be9c 100644 --- a/rep-2011.rst +++ b/rep-2011.rst @@ -329,7 +329,7 @@ Consider the following example: # D.msg bool d_bool -The corresponding ``TypeDescription`` for ``A.msg`` will be as follows: +The corresponding ``TypeDescription`` for ``A.msg`` will be as follows, with the referenced type descriptions accessible as ``IndividualTypeDescription`` types in the ``referenced_type_descriptions`` field of ``A``: .. code:: @@ -339,8 +339,8 @@ The corresponding ``TypeDescription`` for ``A.msg`` will be as follows: C_IndividualTypeDescription, D_IndividualTypeDescription] -With the referenced type descriptions accessible as ``IndividualTypeDescription`` types in the ``referenced_type_descriptions`` field of ``A`` (and also for `A`, its own ``type_description`` field. -In the case where a type description contains no referenced types (i.e., when it has no fields, or all of its fields are primitive types), the ``referenced_type_descriptions`` array will be empty. +Note that the type description for ``A`` itself is found in the ``type_description`` field instead of the ``referenced_type_descriptions`` field. +Additionally, in the case where a type description contains no referenced types (i.e., when it has no fields, or all of its fields are primitive types), the ``referenced_type_descriptions`` array will be empty. .. code:: From 3039877ed9cd14b48362f0e58428a9f1f0b73398 Mon Sep 17 00:00:00 2001 From: methylDragon Date: Thu, 7 Jul 2022 20:14:01 -0700 Subject: [PATCH 4/5] Implement nested introspection API Signed-off-by: methylDragon --- rep-2011.rst | 113 ++++++++++++++++++++++++++++++--------------------- 1 file changed, 67 insertions(+), 46 deletions(-) diff --git a/rep-2011.rst b/rep-2011.rst index eb9e6be9c..e5e502b49 100644 --- a/rep-2011.rst +++ b/rep-2011.rst @@ -488,7 +488,7 @@ The following is an example of how this plugin matching and loading interface co .. code:: // Suppose LaserScanDescription reports that it uses FastCDR v1.0.24 for its serialization - rcl_message_description_t LaserScanDescription = node->get_type_description("/scan"); + rcl_runtime_introspection_description_t LaserScanDescription = node->get_type_description("/scan"); rcl_type_introspection_t * introspection_handle; introspection_handle->init(); // Locate local plugins here @@ -500,79 +500,100 @@ The following is an example of how this plugin matching and loading interface co // If we wanted to force the use of MicroCDR instead introspection_handle->match_plugin(LaserScanDescription->get_serialization_type(), "microcdr"); -Then, the plugin should be able to use the description to deserialize the message buffer using the plugin: +Example Introspection API +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The following is an example for how the introspection API could look like. + +First, certain runtime introspection structs should be created to support introspection, mirroring the type description meta-types laid out in he Type Description Distribution section: + +.. code:: + + struct rcl_runtime_introspection_field_t { // Mirroring Field + void * value; + uint8_t * type; + const char * field_name; + const char * nested_type_name; + } + + struct rcl_runtime_introspection_message_t { // Mirroring IndividualTypeDescription + const char * type_name; + rcl_runtime_introspection_field_t ** fields; + }; + +With these structs, the plugin from the previous section should be able to use the description to introspect the message buffer directly: .. code:: - rcl_deserialized_message_t * scan_msg; - introspection_handle->deserialize(&plugin, message_buffer, &LaserScanDescription, scan_msg); + rcl_runtime_introspection_field_t * scan_time_field = introspection_handle->get_field_from_buffer_by_name( + &plugin, message_buffer, &LaserScanDescription, "scan_time" + ); + if (scan_time_field->type == RCL_RUNTIME_INTROSPECTION_UINT8_T) { + float scan_time = *((float*) scan_time_field->value); + } // Where, internally... - // rcl_type_introspection_t->(*deserialize) - void deserialize(rcl_serialization_plugin_t *plugin, - void *buffer, - rcl_message_description_t *description, - rcl_deserialized_message_t *msg) + // rcl_type_introspection_t->(*get_field_from_buffer_by_name) + rcl_runtime_introspection_field_t * get_field_from_buffer_by_name( + rcl_serialization_plugin_t *plugin, void + *buffer, + rcl_runtime_introspection_description_t *description, + const char * field_name) { ... // Do something serialization specific - plugin_internal_description_t parsed_description = plugin->impl->parse_description(description); - plugin->impl->deserialize(buffer, parsed_description, msg); ... + + plugin_internal_description_t parsed_description = plugin->impl->parse_description(description); + return plugin->impl->get_field_from_buffer_by_name(buffer, parsed_description, field_name); } -Similar interfaces could be created to allow access a message's fields by name or index without deserializing the whole message. +.. TODO:: (methylDragon) I realized that parsing the description on each get_field/deserialization call can get quite expensive. It might be good to store the parsed description somehow? Maybe like a lookup table that's lazily initialized in each plugin object? Or a bounded lookup table?? This might just be an implementation detail though, or too early for optimization, so I'll ignore this for now. -Example Introspection API -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Once the serialization library plugins are able to deserialize the raw message buffer, downstream programs can then introspect the constructed deserialized message object, which should be laid out as such: +Or it could obtain the entire message introspection struct, and iterate through the fields (though this is slower, since the entire buffer has to be read): .. code:: - struct rcl_deserialized_field_t { - void * value; - char * type; - } - - struct rcl_deserialized_message_t { - int message_field_count; - const char** message_field_names; - const char** message_types; + rcl_runtime_introspection_message_t * scan_msg = introspection_handle->get_message_from_buffer( + &plugin, message_buffer, &LaserScanDescription + ); - // Some dynamically allocated key->value associative map type storing void * field values - rcl_associative_array message_fields; + for (rcl_runtime_introspection_field_t ** field_ptr = scan_msg->fields; *field_ptr != NULL; field_ptr++) { + rcl_runtime_introspection_field_t * field = *field_ptr; - // Function pointers - rcl_deserialized_field_t * (*get_field_by_index)(int index); - }; + // Do whatever... + } -Now, for a given message description `Foo.msg`: +The API should also be able to handle nested fields (to an arbitrary level of nesting). +(The following example, outputs ``rcl_runtime_introspection_field_t`` and ``rcl_runtime_introspection_message_t``, which can be introspected as per the previous examples, or further reduced to lower level nested types recursively.) .. code:: - // Foo.msg - bool bool_field - char char_field - float32 float_field + // Getting the nested field directly from the buffer + rcl_runtime_introspection_field_t * sequence_field = introspection_handle->get_field_from_buffer_by_name( + &plugin, message_buffer, &LaserScanDescription, "header.seq" + ); -The corresponding `rcl_deserialized_message_t` can be queried accordingly: + // Getting an rcl_runtime_introspection_message_t object for the nested field directly from the buffer + rcl_runtime_introspection_message_t * header_message = introspection_handle->get_nested_message_from_buffer_by_name( + &plugin, message_buffer, &LaserScanDescription, "header" + ); -.. code:: + // Getting an rcl_runtime_introspection_message_t object from a rcl_runtime_introspection_field_t + rcl_runtime_introspection_field_t * header_field = introspection_handle->get_field_from_buffer_by_name( + &plugin, message_buffer, &LaserScanDescription, "header" + ); - rcl_deserialized_message_t * foo_msg; - foo_msg->message_field_names[0]; // "bool_field" - foo_msg->message_types[0]; // "bool" + rcl_runtime_introspection_message_t * header_message = introspection_handle->get_nested_message_from_field( + &plugin, &HeaderDescription, header_field + ); - // Get the field - if (strcmp("bool", foo_msg->get_field_by_index(0)->type) == 0) - { - *((bool*)foo_msg->get_field_by_index(0)->value); - } +.. NOTE:: (methylDragon) Getting a nested message from buffer by name is likely to only work on top level nested members!! e.g. RTI Connext DDS doesn't look like it can introspect the buffer for low level nested types without first deserializing the type into a DynamicData object. That's I wrote a way to obtain a lower level message type from a field's ``void * value`` member, which should store said DynamicData object. + +.. NOTE:: (methylDragon) Also, it should be noted that some DDS vendors use data loaning for nested members. So the above introspection API only really works for reading data... This makes it very difficult to maintain that loaning if there are multiple layers of nesting. I think separate interfaces should be created to construct a **new** message if we want to modify any fields, but we should discuss. -.. TODO:: Create pseudocode/definitions for sequences and arbitrarily nested message descriptions. Rationale ========= From 856372de03a2c892201410cc2a1e93edb5fad131 Mon Sep 17 00:00:00 2001 From: methylDragon Date: Thu, 14 Jul 2022 20:14:41 -0700 Subject: [PATCH 5/5] Rework nested introspection API Signed-off-by: methylDragon --- rep-2011.rst | 141 +++++++++++++++++++++++++++++---------------------- 1 file changed, 81 insertions(+), 60 deletions(-) diff --git a/rep-2011.rst b/rep-2011.rst index e5e502b49..fc7ab8b2b 100644 --- a/rep-2011.rst +++ b/rep-2011.rst @@ -498,101 +498,122 @@ The following is an example of how this plugin matching and loading interface co rcl_serialization_plugin_t * plugin = introspection_handle->load_plugin(plugin_name); // If we wanted to force the use of MicroCDR instead - introspection_handle->match_plugin(LaserScanDescription->get_serialization_type(), "microcdr"); + introspection_handle->match_plugin(LaserScanDescription->get_serialization_format(), "microcdr"); Example Introspection API ^^^^^^^^^^^^^^^^^^^^^^^^^ The following is an example for how the introspection API could look like. +This example will show a read-only interface. -First, certain runtime introspection structs should be created to support introspection, mirroring the type description meta-types laid out in he Type Description Distribution section: +Overview +"""""""" + +It should comprise several components +- a handler for the message buffer, to handle pre-processing (e.g. decompression) +- a handler for the message description, to keep track of message field names of arbitrary nesting level +- handler functions for message buffer introspection + +Also, this example uses the LaserScan message definition: https://github.com/ros2/common_interfaces/blob/foxy/sensor_msgs/msg/LaserScan.msg + +.. TODO:: (methylDragon) Add a reference somehow? + +API Example +""""""""""" + +First, the message buffer handler: .. code:: - struct rcl_runtime_introspection_field_t { // Mirroring Field - void * value; - uint8_t * type; - const char * field_name; - const char * nested_type_name; + struct rcl_buffer_handle_t { + const void * buffer; // The buffer should not be modified + + const char * serialization_type; + rcl_serialization_plugin_t * serialization_plugin; + rcl_runtime_introspection_description_t * description; // Convenient to have + + // And some examples of whatever else might be needed to support deserialization or introspection... + void * serialization_impl; } - struct rcl_runtime_introspection_message_t { // Mirroring IndividualTypeDescription - const char * type_name; - rcl_runtime_introspection_field_t ** fields; - }; +The message buffer handler should allocate new memory if necessary, or store a pointer to the message buffer otherwise in its ``buffer`` member. -With these structs, the plugin from the previous section should be able to use the description to introspect the message buffer directly: +Then, functions should be written that allow for convenient traversal of the type description tree. +These functions should allow a user to get the field names and field types of the top level type, as well as from any nested types. .. code:: - rcl_runtime_introspection_field_t * scan_time_field = introspection_handle->get_field_from_buffer_by_name( - &plugin, message_buffer, &LaserScanDescription, "scan_time" - ); - if (scan_time_field->type == RCL_RUNTIME_INTROSPECTION_UINT8_T) { - float scan_time = *((float*) scan_time_field->value); - } + struct rcl_field_info_t { // Mirroring Field + const char * field_name; // This should be an absolute address (e.g. "header.seq", instead of "seq") - // Where, internally... - // rcl_type_introspection_t->(*get_field_from_buffer_by_name) - rcl_runtime_introspection_field_t * get_field_from_buffer_by_name( - rcl_serialization_plugin_t *plugin, void - *buffer, - rcl_runtime_introspection_description_t *description, - const char * field_name) - { - ... + uint8_t type; + const char * nested_type_name; // Populated if the type is not primitive + }; - // Do something serialization specific + // Get descriptions + rcl_runtime_introspection_description_t LaserScanDescription = node->get_type_description("/scan"); + rcl_runtime_introspection_description_t HeaderDescription = node->get_referenced_description(LaserScanDescription, "Header"); - ... + // All top-level fields from description + rcl_field_info_t ** fields = get_field_infos(&LaserScanDescription); - plugin_internal_description_t parsed_description = plugin->impl->parse_description(description); - return plugin->impl->get_field_from_buffer_by_name(buffer, parsed_description, field_name); - } + // A single field from description + rcl_field_info_t * header_field = get_field_info(&LaserScanDescription, "header"); + + // A single field from a referenced description + rcl_field_info_t * stamp_field = get_field_info(&HeaderDescription, "stamp"); + + // A nested field from top-level description + rcl_field_info_t * stamp_field = get_field_info(&LaserScanDescription, "header.stamp"); + +Finally, there should be functions to obtain the data stored in the message fields. +This could be by value or by reference, depending on what the serialization library supports, for different types. -.. TODO:: (methylDragon) I realized that parsing the description on each get_field/deserialization call can get quite expensive. It might be good to store the parsed description somehow? Maybe like a lookup table that's lazily initialized in each plugin object? Or a bounded lookup table?? This might just be an implementation detail though, or too early for optimization, so I'll ignore this for now. +There minimally needs to be a family of functions to obtain data stored in a single primitive message field, no matter how deeply nested it is. +These need to be created for each primitive type. -Or it could obtain the entire message introspection struct, and iterate through the fields (though this is slower, since the entire buffer has to be read): +The rest of the type introspection machinery can then be built on top of that family of functions, in layers higher than the C API. .. code:: - rcl_runtime_introspection_message_t * scan_msg = introspection_handle->get_message_from_buffer( - &plugin, message_buffer, &LaserScanDescription - ); + rcl_buffer_handle_t * scan_buffer = node->get_processed_buffer(some_raw_buffer); - for (rcl_runtime_introspection_field_t ** field_ptr = scan_msg->fields; *field_ptr != NULL; field_ptr++) { - rcl_runtime_introspection_field_t * field = *field_ptr; + // Top-level primitive field + get_primitive_field_float32(scan_buffer, "scan_time"); - // Do whatever... - } + // Nested primitive field + get_primitive_field_uint32_seq(scan_buffer, "header.seq"); + + // Nested primitive field sequence element (overloaded) + get_field_seq_length(scan_buffer, "header.seq"); // Support function + get_primitive_field_uint32(scan_buffer, "header.seq", 0); -The API should also be able to handle nested fields (to an arbitrary level of nesting). -(The following example, outputs ``rcl_runtime_introspection_field_t`` and ``rcl_runtime_introspection_message_t``, which can be introspected as per the previous examples, or further reduced to lower level nested types recursively.) +If we attempt to do the same by reference, the plugin might decide to allocate new memory for the pointer, or return a pointer to existing memory. .. code:: - // Getting the nested field directly from the buffer - rcl_runtime_introspection_field_t * sequence_field = introspection_handle->get_field_from_buffer_by_name( - &plugin, message_buffer, &LaserScanDescription, "header.seq" - ); + // Nested primitive field + get_primitive_field_uint32_seq_ptr(scan_buffer, "header.seq"); + + // Be sure to clean up any dangling pointers + finalize_field(some_field_data_ptr); + +Error cases +""""""""""" - // Getting an rcl_runtime_introspection_message_t object for the nested field directly from the buffer - rcl_runtime_introspection_message_t * header_message = introspection_handle->get_nested_message_from_buffer_by_name( - &plugin, message_buffer, &LaserScanDescription, "header" - ); +The following should be error cases: - // Getting an rcl_runtime_introspection_message_t object from a rcl_runtime_introspection_field_t - rcl_runtime_introspection_field_t * header_field = introspection_handle->get_field_from_buffer_by_name( - &plugin, message_buffer, &LaserScanDescription, "header" - ); +- accessing field data as incorrect type +- accessing or introspecting incorrect/nonexistent field names - rcl_runtime_introspection_message_t * header_message = introspection_handle->get_nested_message_from_field( - &plugin, &HeaderDescription, header_field - ); +.. TODO: (methylDragon) Are there more cases? It feels like there are... -.. NOTE:: (methylDragon) Getting a nested message from buffer by name is likely to only work on top level nested members!! e.g. RTI Connext DDS doesn't look like it can introspect the buffer for low level nested types without first deserializing the type into a DynamicData object. That's I wrote a way to obtain a lower level message type from a field's ``void * value`` member, which should store said DynamicData object. +Pointer lifecycles +"""""""""""""""""" -.. NOTE:: (methylDragon) Also, it should be noted that some DDS vendors use data loaning for nested members. So the above introspection API only really works for reading data... This makes it very difficult to maintain that loaning if there are multiple layers of nesting. I think separate interfaces should be created to construct a **new** message if we want to modify any fields, but we should discuss. +- the raw message buffer should outlive the ``rcl_buffer_handle_t``, since it is not guaranteed that the buffer handle will allocate new memory +- the ``rcl_buffer_handle_t`` should outlive any returned field data pointers, since it is not guaranteed that the serialization plugin will allocate new memory +- however, ``rcl_field_info_t`` objects **do not** have any lifecycle dependencies, since they are merely descriptors Rationale