Add runtime type introspection section #3

methylDragon · 2022-06-01T01:31:03Z

No description provided.

methylDragon · 2022-06-01T01:34:38Z

I just saw the reviews from #1 and #2, let me push some changes

wjwwood · 2022-06-02T18:56:26Z

I changed the base to point to the branch for #2, so that you don't have to keep them in sync. Once it is merged we can change the base back to the target branch and merge this one (after iterating).

wjwwood · 2022-06-10T01:15:10Z

@methylDragon can you rebase this, I had to rename the file due our REP number getting swiped (it's my fault).

Signed-off-by: methylDragon <methylDragon@gmail.com>

methylDragon · 2022-06-10T17:36:15Z

Rebased!

wjwwood

Let's fixup what makes sense here, and then continue with the nested types pr on top.

wjwwood · 2022-07-01T20:53:54Z

rep-2011.rst

@@ -321,7 +321,214 @@ The passed ``rosidl_message_type_support_t`` in the init call can be introspecte
 Runtime Type Introspection
 --------------------------

-.. TODO:: terminology could be better? nothing off the top of my head, just deserves more bike-shedding.
+The ability to detect type mismatches and obtain the type description in the event of a mismatch in the previous two sections enables the ability to do runtime type introspection.


Technically the ability to detect type mismatches doesn't support runtime type introspection. The feature could be used to parse a message from a bagfile using just the description included with it. That neither uses a version hash nor does it require type description distribution.

It's true that we do want to use those two things in conjunction with runtime type introspection, but technically they are orthogonal. So this isn't wrong, we should just reword it a bit with better qualifying statements.

rep-2011.rst

wjwwood · 2022-07-01T21:54:44Z

rep-2011.rst

+In the case where there exists multiple applicable plugins for a particular serialization format, the plugin matching should follow this priority order:
+
+- user specified overrides passed to the matching function
+- defaults defined in the plugin matching function, if applicable, otherwise


Mention the use case for this, i.e. connext's CDR library vs Fast-CDR, which should you use if both are available?

wjwwood · 2022-07-01T22:08:25Z

rep-2011.rst

+The following is an example of how this plugin matching and loading interface could look like, defining new ``rcl`` interfaces; with a plugin wrapping FastCDR v1.0.24 for serialization, with some arbitrary message type `Foo`:
+
+.. code::
+
+    // Suppose FooDescription reports that it uses FastCDR v1.0.24 for its serialization
+    rcl_message_description_t FooDescription = node->get_type_description("/foo_endpoint_name");
+
+    rcl_type_introspection_t * introspection_handle;
+    introspection_handle->init();  // Locate local plugins here
+
+    // Plugin name: "fastcdr_v1_0_24"
+    const char * plugin_name = introspection_handle->match_plugin(FooDescription->get_serialization_format());
+    rcl_serialization_plugin_t * plugin = introspection_handle->load_plugin("fastcdr_v1_0_24");
+
+    // If we wanted to force the use of MicroCDR instead
+    introspection_handle->match_plugin(FooDescription->get_serialization_type(), "microcdr");


Let's try using a concrete example, like /scan and sensor_msgs/msg/LaserScan. (or image or point cloud)

wjwwood · 2022-07-01T22:24:47Z

rep-2011.rst

+Plugin Deserialization
+""""""""""""""""""""""
+
+One way a serialization library plugin for FastCDR could be implemented could be as follows, relying on the serialization library to deserialize the message buffer, but with the provided type description to supply type reflection information:
+
+.. code::
+
+    #include <FastBuffer.h>
+    #include <FastCdr.h>
+
+    // If the parsed description is iterable
+    // rcl_serialization_plugin_t->(*deserialize)
+    void deserialize(void *buffer,
+                     plugin_internal_description_t *description,
+                     rcl_deserialized_message_t *msg)
+    {
+      FastCdr * cdr = new FastCdr(new FastBuffer((char *)buffer, description->buffer_length));
+
+      msg->message_field_names = new char*[description->num_fields];
+      msg->message_types = new char*[description->num_fields];
+
+      for (int i = 0; i < description->num_fields; i++)
+      {
+        const char * type = description->field_types[i];
+
+        // Primitives
+        if (strcmp(type, "bool") == 0)
+        {
+          bool tmp;
+        }
+        else if (strcmp(type, "char") == 0)
+        {
+          char tmp;
+        }
+        ...
+
+        // TODO:: What to do about nested types? A recursive unpacking call with a different description?
+
+        cdr >> tmp;
+        msg->message_fields[description->field_names[i]] = reinterpret_cast<void *>(tmp);
+
+        msg->message_field_names[i] = description->field_names[i];
+        msg->message_types[i] = description->field_types[i];
+      }
+      msg->message_field_count = description->num_fields;
+    }


Based on off-line discussion, let's drop this section.

wjwwood · 2022-07-01T22:25:36Z

rep-2011.rst

+Example Introspection API
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Once the serialization library plugins are able to deserialize the raw message buffer, downstream programs can then introspect the constructed deserialized message object, which should be laid out as such:
+
+.. code::
+
+    struct rcl_deserialized_field_t {
+        void * value;
+        char * type;
+    }
+
+    struct rcl_deserialized_message_t {
+        int message_field_count;
+        const char** message_field_names;
+        const char** message_types;
+
+        // Some dynamically allocated key->value associative map type storing void * field values
+        rcl_associative_array message_fields;
+
+        // Function pointers
+        rcl_deserialized_field_t * (*get_field_by_index)(int index);
+    };
+
+Now, for a given message description `Foo.msg`:
+
+.. code::
+
+    // Foo.msg
+    bool bool_field
+    char char_field
+    float32 float_field
+
+The corresponding `rcl_deserialized_message_t` can be queried accordingly:
+
+.. code::
+
+    rcl_deserialized_message_t * foo_msg;
+    foo_msg->message_field_names[0];  // "bool_field"
+    foo_msg->message_types[0];  // "bool"
+
+    // Get the field
+    if (strcmp("bool", foo_msg->get_field_by_index(0)->type) == 0)
+    {
+        *((bool*)foo_msg->get_field_by_index(0)->value);
+    }


All this needs to be updated with the nested types work.

Will be updated in the next PR

Signed-off-by: methylDragon <methylDragon@gmail.com> Co-authored-by: William Woodall <william+github@osrfoundation.org>

Signed-off-by: methylDragon <methylDragon@gmail.com>

methylDragon changed the base branch from master to evolving_message_types_rep June 1, 2022 01:32

methylDragon force-pushed the ch3/runtime_type_introspection branch 3 times, most recently from ce919d9 to eeff240 Compare June 1, 2022 01:55

wjwwood changed the base branch from evolving_message_types_rep to methylDragon/type_description_distribution June 2, 2022 18:55

methylDragon force-pushed the methylDragon/type_description_distribution branch from 4bbc874 to 5095037 Compare June 2, 2022 20:41

wjwwood deleted the branch wjwwood:evolving_message_types_rep June 10, 2022 01:01

wjwwood closed this Jun 10, 2022

wjwwood reopened this Jun 10, 2022

wjwwood changed the base branch from methylDragon/type_description_distribution to evolving_message_types_rep June 10, 2022 01:02

Add section on runtime type introspection

b1550bf

Signed-off-by: methylDragon <methylDragon@gmail.com>

methylDragon force-pushed the ch3/runtime_type_introspection branch from eeff240 to b1550bf Compare June 10, 2022 17:34

wjwwood approved these changes Jul 1, 2022

View reviewed changes

methylDragon and others added 2 commits July 7, 2022 14:02

Apply REP rephrases

75c13ca

Signed-off-by: methylDragon <methylDragon@gmail.com> Co-authored-by: William Woodall <william+github@osrfoundation.org>

Apply more REP rephrases

bdfa2cf

Signed-off-by: methylDragon <methylDragon@gmail.com>

methylDragon merged commit 22f6cb7 into wjwwood:evolving_message_types_rep Jul 8, 2022

methylDragon deleted the ch3/runtime_type_introspection branch July 15, 2022 02:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add runtime type introspection section #3

Add runtime type introspection section #3

methylDragon commented Jun 1, 2022

methylDragon commented Jun 1, 2022

wjwwood commented Jun 2, 2022

wjwwood commented Jun 10, 2022

methylDragon commented Jun 10, 2022

wjwwood left a comment

wjwwood Jul 1, 2022

methylDragon Jul 8, 2022

wjwwood Jul 1, 2022

methylDragon Jul 8, 2022

wjwwood Jul 1, 2022

methylDragon Jul 8, 2022

wjwwood Jul 1, 2022

methylDragon Jul 8, 2022

wjwwood Jul 1, 2022

methylDragon Jul 8, 2022

Add runtime type introspection section #3

Add runtime type introspection section #3

Conversation

methylDragon commented Jun 1, 2022

methylDragon commented Jun 1, 2022

wjwwood commented Jun 2, 2022

wjwwood commented Jun 10, 2022

methylDragon commented Jun 10, 2022

wjwwood left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment