Why does this project exist?

[Pyrolistical]

I don't understand the motivation of this project. You have described capabilities you want in an identifier, but not why anybody would need them.

It sounds like you want something like https://jwt.io/introduction/ but why should identifiers be parsable generally? Why not just an opaque identifier all the time?

Here's an URL safe identifier for any binary: https://en.wikipedia.org/wiki/Base64#URL_applications

[mattbishop]

I find that in my work I have to take in a string identifier either from a URI or part of a payload, and then figure out how to interpret it. This requires out-of-band communication with either docs, if there is any, or the sender of the string, if I can identify that person or system.

Questions I would have are: What is the string? Is it an encoded value of another type? Should I parse the string value into some data structure?

This project is to see about encoding the answers to these questions into the value itself so one can take an identifier and encode/decode it without having to consult an outside source. It looks a lot like dynamic data transfer protocols like MessagePack, and in fact may end up just being a set of patterns on using MessagePack as an identifier encoding mechanism.

[Pyrolistical]

Sounds like you are trying to fit an json object into an identifier, which begs the question is it really an identifier then?

[mattbishop]

Ha, not exactly, though sometimes people do that exact thing in an identifier, which sometimes has to happen given other constraints. I'm more interested in the kinds of identifiers that are more mundane but don't have good ways of defining them or transporting them.

[Pyrolistical]

You can transport anything that can be serialized to binary. Object -> serialized binary -> compress -> base64, but i would keep this as an internal implementation detail. I would avoid the ability for anybody to be able to parse or reverse engineer an identifier.

If you want discoverability, then I would put discoverable content outside of an identifier, like in JSON/MessagePack

[mattbishop]

Well you kinda just made my case. I have to know out-of-band information about the serialized string in order to process it. If I want to communicate that in the payload, your suggestion is to add fields to the payload describing how to consume the identifier.

I'd like to see if there's a better, self-describing way to do these things.

[Pyrolistical]

I wasn't talking about adding fields to describe how to consume the identifier, I would still treat all identifiers as opaque.

The implementation detail to serializing anything to be an identifier is something I would do to produce identifiers, but its not something I would make as part of my API the consumers of my identifiers. I would never expect my consumers to decompress and deserialize identifiers. They would be fools to do so, as I have am only offering an opaque identifier, any assumptions around it are invalid.

Please describe a concrete use case you one would want to parse an identifier.

[mattbishop]

What you describe as desirable for identifiers (opacity) is the current state of identifiers today. Opacity is fairly useless for information hiding IMO because anyone who wants to inspect the values can decode them with very little effort.

Identifiers need to be generated and consumed, and the problem is communication. Both parties need to agree on how the IDs are going to be structured and encoded. This takes time and is error-prone. If Identifiers were self-describing one would not have to learn how to consume it. Moreover, self-describing identifiers can become multivariate--sometimes it's a number, sometimes it's a UUID, depending on whatever the situation calls for.

Another problem is in the code that manipulates the IDs. I often find code that doesn't remember to decode an ID, or tries to decode it twice. Ditto for the encoding part. An identifiers lib would help them reduce these mistakes with assertions.

[Pyrolistical]

We are just talking about the API around identifiers. I think its an anti-pattern to provide any API around identifiers as that only complicates my system. Who cares if an identifier is a number or UUID? Just treat them all as a String and move on.

All bugs I've seen around third-party identifiers have been attempts to treat them with some logic. I've fixed all those bugs by treating third-party identifiers as just Strings. Why introduce an API around identifiers which people can write bad code for, when you can just enforce opacity?

[mattbishop]

If you are in the middle then you can do exactly this—ignore the strong value. This project is not something you would find helpful.

However if you find yourself at either end of the pipe then opacity is unhelpful. My project is about the ends not the pass-through code.

Given your insight it will be important to make no requirements of routers and anther components so thank you for that.