Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deep merge strategy #1759

Merged
merged 2 commits into from
Jul 30, 2021
Merged

Deep merge strategy #1759

merged 2 commits into from
Jul 30, 2021

Conversation

yorinasub17
Copy link
Contributor

@yorinasub17 yorinasub17 commented Jul 29, 2021

This implements deep merge strategy for include. Refer to the updated docs for more info on what this means.

@yorinasub17 yorinasub17 requested a review from brikis98 as a code owner July 29, 2021 15:51
@yorinasub17 yorinasub17 mentioned this pull request Jul 29, 2021
5 tasks
Copy link
Member

@brikis98 brikis98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! For the most part, LGTM. Just had some NITs and a couple questions.

config/include.go Show resolved Hide resolved
Comment on lines +293 to +295
if sourceConfig.RetryableErrors != nil {
targetConfig.RetryableErrors = append(targetConfig.RetryableErrors, sourceConfig.RetryableErrors...)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work OK if targetConfig.RetryableErrors is nil?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Append handles nil correctly. See https://play.golang.org/p/5RRq6Owj_sS

Comment on lines +320 to +322
// MAINTAINER'S NOTE: The following structs cannot be deep merged due to an implementation detail (they do not
// support nil attributes, so we can't determine if an attribute was intentionally set, or was defaulted from
// unspecified - this is especially problematic for bool attributes).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see... So in the future, we'd need to update these to support nil?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea because we can't tell if a false bool attribute was intentionally set in the code in the child terragrunt.hcl config, or if it was because it was unspecified. If unspecified, then we want to inherit from the parent, but if it was intentionally set, then it needs to override the parent. Since the resulting behavior is different, it's important that we can differentiate the two cases.

I decided to avoid addressing this for now in this PR since it was already getting big, but we will definitely want to address this as it feels limiting to not be able to deep merge an important block like remote_state.

@@ -393,6 +394,63 @@ inputs = {
}
```

**What is deep merge?**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: consider adding an example showing two terragrunt.hcl files and what a merged output would look like.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added example: #1764

When the `merge_strategy` for the `include` block is set to `deep`, Terragrunt will perform a deep merge of the included
config. For Terragrunt config, deep merge is defined as follows:

- For simple types, the source overrides the target.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: What is source and what is target? Would help to clarify, perhaps with the example mentioned above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched to child and parent language: #1764

Comment on lines +419 to +420
Finally, `dependency` blocks have special treatment. When doing a `deep` merge, `dependency` blocks from **both** child
and parent config are accessible in **both** places. For example, consider the following setup:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. Why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really have a good answer to this other than that it naturally happened when I coded the deep merge. I actually realized this fact when I started testing it.

I remember chasing the reasoning when I realized it during implementation, but it's been about 2 months since I coded this up so I have forgotten it now...

@yorinasub17
Copy link
Contributor Author

Thanks for review! I'll merge now, and then open a follow up PR with the docs nit fixes.

@yorinasub17 yorinasub17 merged commit d3af798 into master Jul 30, 2021
@yorinasub17 yorinasub17 deleted the yori-deep-merge branch July 30, 2021 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants