encoding/cuedata: Add CUEdata package

[codewithcheese]

Introduces a new (en|de)coder to covert any CUE file to and from a literal data format. Non concrete expressions in each struct are serialized into a syntax string and added as a field, labelled $$cue.

This allows for CUE files to be stored and transmitted as structured data. Our use case at balena is to store CUE files in a document database and query against the concrete values, the resulting documents can then be decoded back to CUE and evaluated as CUE. I think others would share a similar use case, hence I am creating this PR. If this is not acceptable, we will create it as a standalone tool.

I have named the format CUEdata, but I am open to alternatives. I see in issue #130 QSON as in "Q (CUE) Strict Object Notation" and CUEL as in "CUE Literal" were suggested.

This address #1250, for our needs its a more complete solution than JSONSchema, since it allows the document to contain arbitrarily complex CUE expressions that could not be encoded as JSONSchema.

I am new to the Go language and this is an implementation by imitation, I have probably copied some techniques that the CUE code base would like to move away from. I am not aware of all the Go idioms. Please feel free to let me know any better solutions.

Resolves #1435

[myitcv]

@codewithcheese thank you for raising this PR, we're delighted to welcome new contributions!

Referring to the contribution guide, please can I check where this particular approach/design has been previously agreed?

I can quite understand that you might have wanted to jump in and have a get familiar with the code. At the same time, we want to be respectful of people's time, and discuss changes ahead of time:

Excluding very trivial changes, all contributions should be connected to an existing issue. Feel free to open one and discuss your plans. This process gives everyone a chance to validate the design, helps prevent duplication of effort, and ensures that the idea fits inside the goals for the language and tools. It also checks that the design is sound before code is written; the code review tool is not the place for high-level discussions.

[codewithcheese]

hey @myitcv, I opened this discussion #1250 but unfortunately it was during the summer holidays and it did not receive much attention at the time. I recently discovered #130, and realized in retrospect, that I should have likewise utilized an issue to explore how to solve the use case we were facing.

[myitcv]

unfortunately it was during the summer holidays and it did not receive much attention at the time

This is a very fair point. Thank you for being patient!

We absolutely don't want to discourage contributions by referring people to the contributing guide. Instead we want to be as sensitive and respectful of people's time, as I mentioned above.

So my suggestion would be as follows: let's either reuse #130, convert #1250 to an issue, or create a new issue (as you see fit) to discuss in more detail what you're proposing, the use case motivating it, how existing mechanisms fall short etc. The reason I stress an issue here is that our workflow for proposals is issue-based, not discussions. That might seem like a small thing, but with lots of issues/discussions, we need some process.

If we go quiet, please feel free to ping us with @ mentions. Whilst I'm not trying to make excuses, it's always a challenge trying to balance the many demands of an open source project, and so therefore pinging people (like on the Go project when issues or CLs go quiet) is entirely reasonable and expected.

Thank you again for your contribution, but above all else your patience and understanding during the summer months. I will help to move along whatever issue discussion so we get a quick decision on next steps.