Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and test merges with MapSlice #331

Closed
wants to merge 1 commit into from
Closed

Fix and test merges with MapSlice #331

wants to merge 1 commit into from

Conversation

hjdr4
Copy link

@hjdr4 hjdr4 commented Feb 16, 2018

This patch adds tests and working code when decoding merges into MapSlice.

@lox
Copy link

lox commented Mar 12, 2018

Hi @hjdr4! Are you able to resolve the conflicts?

Is this a fix for #91?

@hjdr4
Copy link
Author

hjdr4 commented Mar 13, 2018

Hello,

I have resolved the conflicts.
I have tested the snippet in #91 and it outputs what is expected.
It also fixes #184.

@lox
Copy link

lox commented Mar 13, 2018

Thanks! Could you explain what the issue was? Might this change effect anything else?

@hjdr4
Copy link
Author

hjdr4 commented Mar 14, 2018

MapSlice allows to keep ordering when decoding YAML and this is sometimes required.
The problem is already described in the referenced issues : it does not work correctly at the moment.

So first I wrote a test that shows it's broken (using already defined test data) and code that does what is expected.

I didn't touch the others tests and they are all passing so I guess this won't break anything.

@lox
Copy link

lox commented Mar 19, 2018

@rogpeppe any thoughts on this one?

@niemeyer
Copy link
Contributor

@lox Any thoughts from you on it? :)

@lox
Copy link

lox commented Mar 19, 2018

I'm still getting my head around the impact of the addition of the recursive merge, but it seems to be a clean patch with tests that addresses these specific issues.

@lox
Copy link

lox commented Mar 19, 2018

I'm just in the process of testing it.

@lox
Copy link

lox commented Mar 19, 2018

👍 from me.

@niemeyer
Copy link
Contributor

There are assumptions about typing in the general logic of merge which may not be fulfilled by the structure, in the sense that slices won't necessarily have that type simply because their kind is a slice.

All, I appreciate your help with this, but I'd like to put a tombstone on this issue. This is adding logic to general code paths in v2 which may break down even when not using MapSlice, and I don't want to look back into that. MapSlice will die in v3, and be replaced by a much more flexible intermediate representation. I want to focus on that instead, and anyone that really wants such MapSlices and merges, which is not quite common, can move forward and migrate into it.

Again, thanks for your time here.

@niemeyer niemeyer closed this Mar 26, 2018
@lox
Copy link

lox commented Mar 26, 2018

No problems @niemeyer, looking forward to v3!

@hjdr4
Copy link
Author

hjdr4 commented Mar 26, 2018

Those assumptions are already done https://github.com/go-yaml/yaml/blob/v2/decode.go#L585 and https://github.com/go-yaml/yaml/blob/v2/decode.go#L653.

If you want a stronger test, it's just a matter of changing the merge() test to

out.Kind() == reflect.Slice && out.Type().Elem() == mapItemType 

@niemeyer
Copy link
Contributor

@hjdr4 The logic in merge() itself is general, and called from multiple places. Nothing in there checks the type of the slice.

To be very clear, though, I'm not blaming you and this is a normal issue that we could easily sort out in reviews. But given yaml.v3 is close by and all of that will be dead, I'd indeed prefer to just not touch the logic that is there for several years and instead support any use cases on v3.

yob added a commit to buildkite/agent that referenced this pull request Mar 26, 2021
We're were using a fork of gopkg.in/yaml.v2 (v2.1.1) with a patch to fix
ordering of Maps when used in conjunction with named anchors (this PR
[1]).

Upstream is now at v2.4.0, however the Maps+anchors patch hasn't been
merged. We've updated our fork to be v2.4.0+the patch, and this will
start using that new version.

The list of changes between v2.1.1 and v2.4.0 is fairly small [2], but
it includes a number of parsing bugs that could result in a panic, like
the 3 commits linked at [3]. I've confirmed this fixes a panic reported
by one of our customers while uploading a pipeline.yml.

[1] go-yaml/yaml#331
[2] go-yaml/yaml@v2.1.1...v2.4.0
[3] go-yaml/yaml#323
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants