Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to gopkg.in/yaml.v3 #2468

Merged
merged 7 commits into from
Jul 20, 2022
Merged

Migrate to gopkg.in/yaml.v3 #2468

merged 7 commits into from
Jul 20, 2022

Conversation

colega
Copy link
Contributor

@colega colega commented Jul 19, 2022

What this PR does

Removes all usages of gopkg.in/yaml.v2 because it has some known bugs and we can't just use yaml.v3 for parsing in some places because UnmarshalYAML function signature isn't compatible between boths.

I'd recommend reviewing the commits and their descriptions individually to get a better understanding of the changes.

Notes for the reviewers:

There's no yaml.UnmarshalStrict anymore, so we need to create a new decoder and set the KnownFields flag, which is documented as:

KnownFields ensures that the keys in decoded mappings to exist as fields in the struct being decoded into.

Additionally ,gopkg.in/yaml.v3 implementation doesn't seem to set the entire value to zero value if an empty node is specified in the YAML. So we don't need that check again, and we can adapt the test just to check that this assertion is true.

Which issue(s) this PR fixes or relates to

Not fixing, but related to the common config efforts.

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

colega added 4 commits July 19, 2022 17:24
This replaces all usages of gopkg.in/yaml.v2 by v3 everywhere, adapts
the UnmarshalYAML methods to the new signature, and replaces the
UnmarshalStrict calls by the decoder with KnownFields flag.

util.YAMLMarshalUnmarshal doesn't work because it doesn't support
unmarshaling map[interface{}]interface{}

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
yaml.v3 can't unmarshal to map[interface{}]interface{},
but actually map[string]interface{} should be enough.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
@colega
Copy link
Contributor Author

colega commented Jul 20, 2022

We can't use gopkg.in/yaml.v3 because it does not inherit the KnownFields flag when running the custom unmarshallers by calling Node.Decode, and apparently there's no way to do it

colega added 3 commits July 20, 2022 09:38
This commit is the latest v3 with
go-yaml/yaml#691 cherry-picked on top of it, to
provide support for strict unmarshaling when doing custom unmarshaling
(see go-yaml/yaml#460)

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
There's no such field as
`alertmanager_notification_limits_per_integration`
and the fixed strict parsing has detected this.

How does this test pass in `main`?

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
gopkg.in/yaml.v3 implementation doesn't seem to set the entire value to
zero value if an empty node is specified in the YAML. So we don't need
that check again, and we can adapt the test just to check that this
assertion is true.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
@colega
Copy link
Contributor Author

colega commented Jul 20, 2022

I've forked gopkg.in/yaml.v3 and cherry-picked go-yaml/yaml#691 that fixes the go-yaml/yaml#460 until it's merged, which IMO is a better solution than switching to a completely different YAML parser now.

Copy link
Member

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! Thank you for taking care of this.

@@ -378,7 +380,7 @@ testuser:

differentUserOverride := `
differentuser:
alertmanager_notification_limits_per_integration:
alertmanager_notification_rate_limit_per_integration:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It took me a while to understand why the test was failing now 🤦. This test didn't do strict unmarshaling previously, so it wasn't really testing anything I guess.

Copy link
Contributor

@bboreham bboreham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work!

I would suggest putting part of this comment into the PR description, because it might get hidden when commits are squashed:

gopkg.in/yaml.v3 implementation doesn't seem to set the entire value to
zero value if an empty node is specified in the YAML. So we don't need
that check again, and we can adapt the test just to check that this
assertion is true.

Copy link
Collaborator

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good job!

@colega colega merged commit 03747bf into main Jul 20, 2022
@colega colega deleted the migrate-to-yaml.v3 branch July 20, 2022 09:10
colega added a commit that referenced this pull request Jul 20, 2022
After upgrading to gopkg.in/yaml.v3 in [#2468], @pstibrany noticed that
int 0 is no longer a valid time.Duration value.

I've sent a PR to fix that: go-yaml/yaml#876 and
cherry-picked the change into the branch we're using in Mimir as the
replacement.

[#2468]: #2468

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
colega added a commit that referenced this pull request Jul 20, 2022
* Fix parsing int 0 as duration in yaml

After upgrading to gopkg.in/yaml.v3 in [#2468], @pstibrany noticed that
int 0 is no longer a valid time.Duration value.

I've sent a PR to fix that: go-yaml/yaml#876 and
cherry-picked the change into the branch we're using in Mimir as the
replacement.

[#2468]: #2468

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Fix typo in the comment.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants