Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(encode): ensure quoting is maintained for scalars #115

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

stormqueen1990
Copy link
Member

Currently, when encoding or decoding integers bigger than 64 bytes using sigs.k8s.io/yaml/goyaml.v3, those values are interpreted as unequivocal strings, which causes quotes to be removed. However, those values should be treated as regular numbers since they conform to the integer format in the YAML specification and honour quoting when present in the original input.

Related to kubernetes-sigs/kustomize#5432 and kubernetes-sigs/kustomize#5558
Fixes #108

Currently, when encoding or decoding integers bigger than 64 bytes using
`sigs.k8s.io/yaml/goyaml.v3`, those values are interpreted as unequivocal
strings, which causes quotes to be removed. However, those values should be
treated as regular numbers since they conform to the integer format in the YAML
specification and honour quoting when present in the original input.
@k8s-ci-robot k8s-ci-robot added the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Aug 14, 2024
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 14, 2024
Comment on lines +741 to +742
A4: 0x0000000000000000000000010000000000000000
A5: 0x000000000000000000000000FFFFFFFFFFFFFFFF
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This repo is oriented at yaml which can be round-tripped to json and used in Kubernetes API objects

These bigints would be problematic for Kubernetes as they do not round-trip to json ... I'm on the fence that we should enable this in this library

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My general thought process was that unquoted values are already not round-trippable to JSON when bigger than 64 bytes, but if quoted those should be maintained. The current implementation assumes that if an integer value does not fit a 64-byte integer then it must be a string but that's not exactly true.

Comment on lines +738 to +739
A1: "0x0000000000000000000000010000000000000000"
A2: "0x000000000000000000000000FFFFFFFFFFFFFFFF"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintaining the quoting on A1/A2 is good ... do we also know that YAMLToJSON works properly here? that's what will be used when reading a yaml manifest and sending it to the kube-apiserver

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add a test case for YAMLToJSON

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

understanding all the paths that come through here and what they do with these values today (as well as how kube-apiserver treats submitted yaml values) is important so we make sure we don't break anyone that is currently "working"

I think the test cases would be something like:

unquoted yaml → yaml (round-trip)
unquoted yaml → json (conversion)
quoted yaml → yaml (round-trip)
quoted yaml → json (conversion)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@liggitt I added some test cases to the TestYAMLtoJSON function in yaml_test.go in my last commit 742f5e6, which I believe covers both of these scenarios since it first converts the YAML to JSON, and then converts it back, but please let me know if I misunderstood!

@Red-M
Copy link

Red-M commented Aug 15, 2024

This does not fix #108

Add round-trip tests as requested in code review and fix a few other scenarios
uncovered by the addition of new tests.
@k8s-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: stormqueen1990
Once this PR has been reviewed and has the lgtm label, please assign liggitt for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 16, 2024
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 26, 2024
@stormqueen1990
Copy link
Member Author

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

yaml.Marshal() can not be forced to leave strings quoted
5 participants