Extension slicing: or what to do when things go wrong

[joshmoore]

cF: https://doc.zeroc.com/ice/3.6/client-server-features/slicing-values-and-exceptions

Given this class hierarchy defined for the Ice protocol:

module ex {

  // Some base data type
  class ZString {
       Bytes data;
  };

  // A particular interpretation of that type
  class ZUTF8 extends ZString {
  };

  // Something arbitrarily more complex ...
  class ZHTML extends ZUTF8 {
      Language lang;
  };

};

a ZHTML instance that was serialized would contain an ordered list of its extension specs as:

In[1]: print(ZHTML().ice_ids())
Out[1]: ["::ex::ZHTML", "::ex::ZUTF8", "::ex::ZString", "::Ice::Object"]

A client which wanted to deserialize this object but did not have access to the ZHTML spec would slice off that interpretation and try the next one. If the only spec that was available was ZString, then that instance would be of type ZString and the client would use the regular handling for that type.

Types that might benefit from this mechanism:

• complex numerics (e.g. extends LongPair if the spec doesn't support the more flexible Long[2])
• timestamps (e.g. extends String)
• Perhaps WIP: Multiscale use-case #23 (e.g. Chunk[] of different sizes)

[tam203]

So I'm not sure about the best implementation but I like the idea of being able to 'descend' from high objects to lower (or even primitives) when not available. The example of dates going to strings (or int's depending on implementation) is a good one.

I'm nervous (but equally not sure there is a better option) about how the spec is aiming to be very light/minimal with extensions handling many/most of the more complex work. My concern is each institute/individual uses their own combination of filters/compressors/etc and that data becomes less interpretable as everyone is using different combinations that don't necessarily port to different languages/versions and data will rust and degrade as this myriad of libraries aren't maintained or updated.

I realise the above paragraph is all "grump" and no solutions but I think this issue (if implemented well) could help alleviate some of this.

👏

[jstriebel]

Atm the v3 core spec provides the idea of a fallback data-type and specifically states behavior around required- and non-required extensions. I believe this is a great leap forward regarding this issue, should we regard this as resolved? cc @joshmoore @tam203

[joshmoore]

Do you imagine that would work in a chain, @jstriebel? Using the original example, if ZHTML specifies ZUTF8 as a fallback type and ZUTF8 defines ZString as the fallback type and if that leads to a user without ZHTML and ZUTF8 being installed to get ZString, then I agree, this is covered.

[jstriebel]

Do you imagine that would work in a chain, @jstriebel? Using the original example, if ZHTML specifies ZUTF8 as a fallback type and ZUTF8 defines ZString as the fallback type and if that leads to a user without ZHTML and ZUTF8 being installed to get ZString, then I agree, this is covered.

Hm, after re-reading this section, I think the current fallback is only meant to provide a single core data-type, in the extreme case this would be r* (see the data type section here). I can see two options to have this chain of fallbacks:

• One might supply a list of fallbacks instead of/in addition to a single value. The last one must be a core dtype, still.
• The data type implementations propose appropriate fallbacks. Then an implementation would still need to know about the originally requested dtype extension, but could implement a fallback for it.

I tend to the first version, it doesn't seem to add too much complexity to the spec IMO. I added a short PR for this change, feel free to comment there directly: #167
The only question left there from my side is if we want those fallback chains across extensions or not.

[jstriebel]

@joshmoore Do you think we can close this since #167 is merged or is there anything else missing?

[sanketverma1704]

As discussed during the ZEP meeting on 12/15, closing this.