Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make json-iter decoder able to recognize duplicate fields #5

Open
luxas opened this issue Aug 30, 2021 · 1 comment
Open

Make json-iter decoder able to recognize duplicate fields #5

luxas opened this issue Aug 30, 2021 · 1 comment

Comments

@luxas
Copy link
Owner

luxas commented Aug 30, 2021

A common way for users to shoot themselves in the foot with YAML (unfortunately) is to misspell something (i.e. pointing to an "unknown field"), indent something at the wrong level (very hard to see this clearly in a large file) (again, becoming an "unknown field" at the wrong context) or register two duplicate fields.

As can be read in YAML 1.2 spec :

JSON's RFC4627 requires that mappings keys merely “SHOULD” be unique, while YAML insists they “MUST” be. Technically, YAML therefore complies with the JSON spec, choosing to treat duplicates as an error. In practice, since JSON is silent on the semantics of such duplicates, the only portable JSON files are those with unique keys, which are therefore valid YAML files.

Due to this; I'm of the opinion that we should treat duplicate fields as errors (or at least, warnings, to begin with) in JSON, too. yaml.v3 errors out if there are duplicate fields already. It can be very non-obvious how parsing is done if duplicate fields don't yield an error, e.g. encoding/json where duplicate fields with object values are merged, whereas all other types are replaced with the latest value (ref: golang/go#24415 (comment))

I coded a sample implementation of duplicate fields error support in upstream json-iterator: https://github.com/luxas/json-iterator/tree/error_for_duplicates

But, I think it should be possible to write this as a json-iter extension as well; that might be the way to go for the moment.
If json-iter supported "warnings", it'd also be possible to begin the transition to duplicate and unknown fields erroring by showing the user warnings (in the spirit of https://kubernetes.io/blog/2020/09/03/warnings/)

@luxas
Copy link
Owner Author

luxas commented Aug 30, 2021

Two examples of real-world problems this may cause:

image
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant