Improve set detailed diffs #2451
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
t0yv0
reviewed
Oct 29, 2024
t0yv0
reviewed
Oct 29, 2024
t0yv0
reviewed
Oct 29, 2024
t0yv0
approved these changes
Oct 29, 2024
VenelinMartinov
added a commit
that referenced
this pull request
Oct 29, 2024
This PR changes the logic in the detailed diff v2 calculation to short-circuit on encountering nils and unknowns. Previously, when we encountered a null or unknown in either news or olds we would still recurse down and compare the non-nil values versus a nil value in order to get any replacements which might be happening. We would then simplify the diff later to trim these extra diffs but would propagate the replacement to the right level. We would now instead short-circuit on encountering a null or unknown and walk the non-null value to find any properties which can trigger a replacement. This makes it much easier to handle sets in #2451 as recursing into sets is not necessary, as they are only compared by hashes. This change is not meant to change any behaviour and is tested in #2405 Stacked on #2515 and #2516
|
I've refactored makeSetDiff into 3 functions and added tests for each one to address https://github.com/pulumi/pulumi-terraform-bridge/pull/2451/files#r1806996735 LMK if you have any comments on that! |
t0yv0
reviewed
Oct 29, 2024
t0yv0
reviewed
Oct 29, 2024
|
We're in code gold territory. I think we can ship this. It's a little clearer, it could be clearer still, but the test coverage is good so that's what matters. Thank you for labeling ints to distinguish hashes and indices. |
iwahbe
approved these changes
Oct 30, 2024
|
This PR has been shipped in release v3.94.0. |
This was referenced Oct 31, 2024
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change adds improved TF set handling to the detailed diff v2. The main challenge here is that pulumi does not have native sets, so set types are represented as lists.
Diffing sets using the hash
To correctly find the diff of two sets we calculate the hash of each element ourselves and do the diffs based on that. What makes this somewhat non-trivial is that due to MaxItemsOne flattening we can't just hash the pulumi
PropertyValues given to us by the engine. Instead we usemakeSingleTerraformInputto transform the values using the schema. We then use the hashes of the elements in the set to calculate the diffs. This allows us to correctly account for shuffling and duplicates, matching the terraform behaviour of sets.When returning the element indices back to the engine, we need to return them in terms of oldState and newInputs because the engine does not have access to the plannedState (see #2281). To do that we keep the newInputs and match plannedState elements to their newInputs index by the set hash. Note that this is not possible if the set element contains any computed properties - these can change during the planning process, so we do not attempt to match and print a warning about possibly inaccurate diff results instead.
Unknowns in sets
Note that the terraform planning process does not allow a set to contain any unknowns, because that breaks the hashing. Because of that plan should always return an unknown for a set which contains any unknowns. This accounts for cases where resolving the unknown can result in duplicate elements.
Unknown elements in sets - the whole set becomes unknown in the plan, so the algorithm no longer works. Currently we return an update for the whole set to the engine and it does the diff on its side.
Testing
This PR also includes tests for the set behaviour, both unit tests for the detailed diff algorithm and integration tests using pulumi programs for:
Issues
Builds on #2405
Stacked on #2515, #2516, #2496 and #2497
fixes #2200
fixes #2300
fixes #1904
fixes #186