Cyclone and Foxglove possible interop issue for complex appendable types

[max-inn]

Description
I believe appendable annotation is part of the supported OMGIDL features as I have not come across anything on the readme saying otherwise. If that's not the case then this issue is not relevant.

I noticed a possible interop issue between Foxglove and cyclonedds. We use cyclonedds v0.10.2 and IDL structs serialized with D_CDR2_LE/BE (I believe internally you call it RTPS_DELIMITED_CDR2_LE/BE) fail to deserialize with something like the following:
RangeError: Offset is outside the bounds of the DataView in field a of Inner at location 24. in field inners of Outer at location 24.

I believe this is due to a mismatch in the implementation of the spec as noted in the following issue eclipse-cyclonedds/cyclonedds-cxx#218

• Version: omgidl-parser/v1.0.3
• Platform: Ubuntu 22.04

Steps To Reproduce
The following IDL fails to deserialize when the serialized data is generated by cyclone:

@appendable
struct Inner {
  uint32 a;
};
@appendable
struct Outer {
  sequence<Inner> inners;
};

Specifically cyclone will serialize Outer with an empty sequence as:
[0, 9, 0, 0, 8, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0] while Foxglove seems to expect: [0, 9, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0]. I validated this by removing the additional sequence header and Foxglove was able to deserialize the data.

Expected Behavior
I would expect Foxglove to be able to deserialize cyclonedds data.

If the difference in the implementation is the issue, do you think it's possible for Foxglove to align on what cyclone does in this case ? I understand there's different interpretations of the spec so I mainly want to know if this is something I need to address myself rather than wait for a new version.

[jtbandes]

Thanks for the report, we will look into this issue soon. Any chance you can share some sample code you used to generate the data?

[defunctzombie]

Internal tracking issue: https://linear.app/foxglove/issue/FG-8929/cyclone-and-foxglove-possible-interop-issue-for-complex-appendable

[max-inn]

Thanks for the quick response.

Thanks for the report, we will look into this issue soon. Any chance you can share some sample code you used to generate the data?

I've forked the cyclonedds python repo and prepared a minimal example that generates the serialized CDR data I've provided above here.

You have to follow the README instructions to get cyclonedds installed in your python env, then you should be able to run the example from the root of the repo with:
python3 examples/foxglove/foxglove.py

Which should print the serialize data as:
[0, 9, 0, 0, 8, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0]

[snosenzo]

@max-inn Thanks for bringing this up, I have a draft PR that at least fixes the issue with the specific example you mention and doesn't break any of our tests. I still have to test it against some other data in our app, but if that works out hopefully we'll be able to have a fix in the app for you in the next week or so.

[max-inn]

That sounds great, thank you !

[snosenzo]

The change has been merged in Foxglove now, it will be in the web deploy by tomorrow and will be in the next desktop release. I'm going to close this issue for now but feel free to reopen or open a new one if you encounter any more issues!