Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(terraform): eval submodules #6411

Merged
merged 5 commits into from
Apr 4, 2024

Conversation

nikpivkin
Copy link
Contributor

@nikpivkin nikpivkin commented Mar 28, 2024

Description

Related issues

Checklist

  • I've read the guidelines for contributing to this repository.
  • I've followed the conventions in the PR title.
  • I've added tests that prove my fix is effective or that my feature works.
  • I've updated the documentation with the relevant information (if needed).
  • I've added usage information (if the PR introduces new options)
  • I've included a "before" and "after" example to the description (if the PR is a user interface change).

@nikpivkin nikpivkin force-pushed the tf-eval-submodules branch from 3d2d5d9 to e64d935 Compare March 28, 2024 11:34
@nikpivkin nikpivkin marked this pull request as ready for review March 28, 2024 12:03
@nikpivkin nikpivkin requested a review from simar7 as a code owner March 28, 2024 12:03
var modules terraform.Modules
for _, definition := range e.loadModules(ctx) {
submodules, outputs, err := definition.Parser.EvaluateAll(ctx)
if err != nil {
e.debug.Log("Failed to evaluate submodule '%s': %s.", definition.Name, err)
continue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why this didn't work as intended, is it because we weren't definition.Parser.Load(ctx) for each definition? Like so: https://github.com/aquasecurity/trivy/pull/6411/files#diff-b10704f6636c4e99c08df82aeb21c2283a75a61953d50b6f800289dbfa44979eR187

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modules are loaded and evaluated in alphabetical order. But modules can have different order and cyclic relationships, so we need to evaluate each module several times. Load is just a new function to load modules once, which returns an evaluator.

assert.Equal(t, "test_value", attr.GetRawValue())
}

func TestCyclicModules(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a nice test case! I'm curious what logic prevents the cyclic loop? In this case module1 uses module2 and vice-versa but how do we figure out to resolve them both?

does the logic still hold true if we have 3 modules in a cycle?

Copy link
Contributor Author

@nikpivkin nikpivkin Mar 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cycling between modules is allowed. Modules are evaluated until the output variables of all modules stop changing or the maximum number of evaluations is reached. In short, if at least one output variable of any module has been updated, we have one unevaluated module and therefore continue the evaluation.

Let's look at an example:

module "module1" {
	source = "./modules/too"
	test_var = module.module2.test_out
}

module "module2" {
	source = "./modules/bar"
	test_var = module.module3.test_out
}

module "module3" {
	source = "./modules/foo"
}

The first evaluation step will completely compute module3, the second module2, and the third module1.

@fwereade
Copy link
Contributor

fwereade commented Apr 1, 2024

Hi @nikpivkin, thank you for working on this – I opened a PR against your branch which I hope will be helpful.

@CLAassistant
Copy link

CLAassistant commented Apr 2, 2024

CLA assistant check
All committers have signed the CLA.

@simar7 simar7 self-requested a review April 3, 2024 00:40
@simar7
Copy link
Member

simar7 commented Apr 3, 2024

lgtm, looks like we need to fix some merge conflicts @nikpivkin

@nikpivkin
Copy link
Contributor Author

@simar7 Fixed

@simar7 simar7 added this pull request to the merge queue Apr 4, 2024
Merged via the queue into aquasecurity:main with commit 13190e9 Apr 4, 2024
12 checks passed
fl0pp5 pushed a commit to altlinux/trivy that referenced this pull request May 6, 2024
Co-authored-by: William Reade <william@stacklet.io>
@nikpivkin nikpivkin deleted the tf-eval-submodules branch June 19, 2024 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug(terraform): module output values are not passed into the context of other modules
4 participants