metaspec

[markus2330]

@kodebach wrote in #3549:

I like were the decision on metadata is going, but I think we might need another more in depth discussion and should maybe move it to a separate PR.

Yes, good idea. Let us wrap #3549 up (with the next meetings) and move the spec and API decisions to the next PR. Discussion can go on here.

AFAICT it we are basically going in the direction of separating the two uses of metadata: specification, and inferred meta-information.

Exactly! Maybe even more uses: built-in specifications and the vendor-specific specifications coming from specload. But I know that you do not like this idea so much 😉

IMO the best solution would be, if plugins like network and shell would take their specifications (e.g. check/ipaddr and execute/set) directly from the spec:/ keys, without the spec plugin having to copy everything to all namespaces. This would however be a massive change, so it is probably out of scope. It is also unclear, how the spec:/ keys would even be passed to non-global plugins.

Yes, I also do not see a way. The spec plugin is basically the mechanism to pass the information. If it would only update references it shouldn't be too expensive anyway...

The next best thing would be, if the spec plugin copied the metadata into meta-keys with namespace metaspec:/. So a Key can have meta:/ and metaspec:/ metadata. The meta:/ keys are the one inferred from the storage file or from the specification, and the metaspec:/ keys are the actual specification. This would still require updating many plugins, but it would be a quite simple change (just adding the metaspec:/ prefix shouldn't be too hard). This also avoids the dual meaning of e.g. array; metaspec:/array=#0 is "default size 1" and meta:/array=#0 is "actual size is 1". It would also make the spec plugin a bit simpler. For example spec doesn't need to handle merging type from spec:/ keys and other namespaces. [1] Instead the type plugin could handle that.

I love the idea, it is ingenious even after thinking about it several times 💖. You invented something what we could call meta-namespaces or meta-cascading and I think this would fit very nicely to the general concepts of Elektra. Of course we need to get all details and implications right, and there are many.

For example: { "mykey": 4 } produces a key mykey=4 with metadata type=long_long. If there is also a specification where mykey has type=int, this wouldn't be a problem. The type plugin would just do the normal validation between metaspec:/type=int and the value 4 and then do an additional check that a whether meta:/type can be assigned to metaspec:/type (e.g. int can be assigned to long_long).

Assuming that applications don't need direct access to the metaspec:/ metadata, the metaspec:/ metadata could also be discarded at the end of kdbGet and regenerated at the start of kdbSet to avoid any inconsistencies between the specification in the spec:/ namespace and the metaspec:/ metadata.

---

[1] spec would not only copy to metaspec:/ it also still needs to create some meta:/ metadata (e.g. type and array) in some cases.

So a very rough sketch to check if we are still on the same page:

• spec would not destroy metadata but only add to other meta-namespaces
• type checking additionally involves a consistency check between spec:/type metaspec:/type ...
• to get the most specific type, a meta-cascading /type can be used (assuming we find a way to order all sources of metadata by priority)

[kodebach]

So a very rough sketch to check if we are still on the same page: [...]

Yes, that seems about right. But I'm not sure about the meta-cascading stuff. Reusing the same KEY_NS_CASCADING would require tagging KeySets as either metadata or normal data. If the type for both is still KeySet it would become confusing, if they behave differently.

[markus2330]

Yes, we need to avoid reusing, so we would also have a KEY_NS_META_CASCADING. Or we avoid passing the meta-keyset to the user, then this would be an internal problem.

[stale]

I mark this issue stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping the issue by writing a message here or create a new issue with the remainder of this issue.
Thank you for your contributions 💖

[markus2330]

We can also drop the idea of meta-cascading and only add the namespace metaspec:. Meta-cascading is imho only useful with more then two sources, i.e. vendor-specific overwrites of specifications.

[github-actions]

I mark this stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping by writing a message here or create a new issue with the remainder of this issue.
Thank you for your contributions 💖

[github-actions]

I closed this now because it has been inactive for more than one year. If I closed it by mistake, please do not hesitate to reopen it or create a new issue with the remainder of this issue.
Thank you for your contributions 💖