fix(terraform): panic when `for-each` contains an object with different attr types #5752

nikpivkin · 2023-12-06T18:14:50Z

Discussed in #5750

^{Originally posted by rwunderer December 7, 2023}

Description

I have a for_each loop in terraform, calling a module multiple times, with different parameters. One of the parameters is an array of strings that may be empty.

So the first for_each run sets eg path_writers = ["somestring"] the second run sets path_writers = []. This is perfectly valid from both terraform's and the modules point of view.

Trivy however gets confused because it seems to build an internal data structure from the first element and expects this parameter to always be a non-empty array. As a result trivy panics.

The same code gets scanned correctly by previous versions of trivy, this bug appeared fresh in 0.48.0
The same issue happens if the parameter in question is a map, not an array. I think it will happen for any complex data type.

Desired Behavior

Trivy completes successfully (reporting misconfigurations or not, depending on the code)

Actual Behavior

Trivy aborts with a panic

Reproduction Steps

1. create a simple `main.tf` with the following code:


module "backups" {
  source       = "../modules/s3_archive_bucket"
  bucket_name  = each.key
  client       = each.value.client
  path_writers = each.value.path_writers

  for_each = {
    "bucket1" = {
      client       = "client1"
      path_writers = ["writer1"]
    },
    "bucket2" = {
      client       = "client2"
      path_writers = []
    }
  }
}

run trivy config .



### Target

Filesystem

### Scanner

Misconfiguration

### Output Format

None

### Mode

Standalone

### Debug Output

```bash
$ trivy config --debug .
2023-12-06T17:07:46.644Z        DEBUG   Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
2023-12-06T17:07:46.653Z        DEBUG   cache dir:  /home/rwunderer/.cache/trivy
2023-12-06T17:07:46.653Z        INFO    Misconfiguration scanning is enabled
2023-12-06T17:07:46.653Z        DEBUG   Policies successfully loaded from disk
2023-12-06T17:07:46.653Z        DEBUG   Enabling misconfiguration scanners: [azure-arm cloudformation dockerfile helm kubernetes terraform terraformplan]
2023-12-06T17:07:46.669Z        DEBUG   The nuget packages directory couldn't be found. License search disabled
2023-12-06T17:07:46.674Z        DEBUG   Walk the file tree rooted at '.' in series
2023-12-06T17:07:46.674Z        DEBUG   Scanning Terraform files for misconfigurations...
2023-12-06T17:07:46.674Z        DEBUG   [misconf] 07:46.674864517 terraform.scanner                Scanning [&{%!s(*mapfs.file=&{ [] {. 256 2147484096 {13930839230419331640 739694060 0xd25ac60} <nil>} {{{0 0} {[] {} 0xc0032de600} map[main.tf:0xc0001bbf60] 0}}}) .}] at '.'...
2023-12-06T17:07:46.679Z        DEBUG   [misconf] 07:46.679107887 terraform.scanner.rego           Overriding filesystem for policies!
2023-12-06T17:07:46.741Z        DEBUG   [misconf] 07:46.741636489 terraform.scanner.rego           Loaded 189 policies from disk.
2023-12-06T17:07:46.742Z        DEBUG   [misconf] 07:46.742434745 terraform.scanner.rego           Overriding filesystem for data!
2023-12-06T17:07:47.323Z        DEBUG   [misconf] 07:47.323100106 terraform.scanner                Scanning root module '.'...
2023-12-06T17:07:47.323Z        DEBUG   [misconf] 07:47.323153797 terraform.parser.<root>          Setting project/module root to '.'
2023-12-06T17:07:47.323Z        DEBUG   [misconf] 07:47.323187071 terraform.parser.<root>          Parsing FS from '.'
2023-12-06T17:07:47.323Z        DEBUG   [misconf] 07:47.323247060 terraform.parser.<root>          Parsing 'main.tf'...
2023-12-06T17:07:47.323Z        DEBUG   [misconf] 07:47.323539517 terraform.parser.<root>          Added file main.tf.
2023-12-06T17:07:47.323Z        DEBUG   [misconf] 07:47.323600288 terraform.parser.<root>          Evaluating module...
2023-12-06T17:07:47.323Z        DEBUG   [misconf] 07:47.323872194 terraform.parser.<root>          Read 1 block(s) and 0 ignore(s) for module 'root' (1 file[s])...
2023-12-06T17:07:47.323Z        DEBUG   [misconf] 07:47.323928402 terraform.parser.<root>          Added 0 variables from tfvars.
2023-12-06T17:07:47.323Z        DEBUG   [misconf] 07:47.323947795 terraform.parser.<root>          Error loading module metadata: open .terraform/modules/modules.json: file does not exist.
2023-12-06T17:07:47.323Z        DEBUG   [misconf] 07:47.323975896 terraform.parser.<root>          Working directory for module evaluation is '/home/rwunderer/caprisys/terraform/scaleway/trivy'
2023-12-06T17:07:47.324Z        DEBUG   [misconf] 07:47.324069666 terraform.parser.<root>.evaluator Filesystem key is '8142758bef01a3284c573f90bf3f87c89aff6501ead5514d662eaac9bc4fc070'
2023-12-06T17:07:47.324Z        DEBUG   [misconf] 07:47.324084296 terraform.parser.<root>.evaluator Starting module evaluation...
panic: inconsistent map element types (cty.Object(map[string]cty.Type{"bucket_name":cty.String, "client":cty.String, "for_each":cty.Object(map[string]cty.Type{"bucket1":cty.Object(map[string]cty.Type{"client":cty.String, "path_writers":cty.Tuple([]cty.Type{cty.String})}), "bucket2":cty.Object(map[string]cty.Type{"client":cty.String, "path_writers":cty.EmptyTuple})}), "id":cty.String, "path_writers":cty.EmptyTuple, "source":cty.String}) then cty.Object(map[string]cty.Type{"bucket_name":cty.String, "client":cty.String, "for_each":cty.Object(map[string]cty.Type{"bucket1":cty.Object(map[string]cty.Type{"client":cty.String, "path_writers":cty.Tuple([]cty.Type{cty.String})}), "bucket2":cty.Object(map[string]cty.Type{"client":cty.String, "path_writers":cty.EmptyTuple})}), "id":cty.String, "path_writers":cty.Tuple([]cty.Type{cty.String}), "source":cty.String}))

goroutine 1 [running]:
github.com/zclconf/go-cty/cty.MapVal(0xc0034d0b28)
        /home/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.13.0/cty/value_init.go:220 +0x40e
github.com/aquasecurity/trivy-iac/pkg/scanners/terraform/parser.(*evaluator).expandBlockForEaches(0xc000ba6480, {0xc002a8e178, 0x1, 0x0?})
        /home/runner/go/pkg/mod/github.com/aquasecurity/trivy-iac@v0.7.0/pkg/scanners/terraform/parser/evaluator.go:325 +0x4df
github.com/aquasecurity/trivy-iac/pkg/scanners/terraform/parser.(*evaluator).expandBlocks(0x78dc240?, {0xc002a8e170?, 0x78dc240?, 0xc0025c39b0?})
        /home/runner/go/pkg/mod/github.com/aquasecurity/trivy-iac@v0.7.0/pkg/scanners/terraform/parser/evaluator.go:207 +0x30
github.com/aquasecurity/trivy-iac/pkg/scanners/terraform/parser.(*evaluator).EvaluateAll(0xc000ba6480, {0x9b30ee8, 0xc000b95b20})
        /home/runner/go/pkg/mod/github.com/aquasecurity/trivy-iac@v0.7.0/pkg/scanners/terraform/parser/evaluator.go:152 +0x2e5
github.com/aquasecurity/trivy-iac/pkg/scanners/terraform/parser.(*Parser).EvaluateAll(0xc0027e38c0, {0x9b30ee8, 0xc000b95b20})
        /home/runner/go/pkg/mod/github.com/aquasecurity/trivy-iac@v0.7.0/pkg/scanners/terraform/parser/parser.go:313 +0x6b7
github.com/aquasecurity/trivy-iac/pkg/scanners/terraform.(*Scanner).ScanFSWithMetrics(0xc00331bf00, {0x9b30ee8, 0xc000b95b20}, {0x9a70260?, 0xc003527f68}, {0x9a58958, 0x1})
        /home/runner/go/pkg/mod/github.com/aquasecurity/trivy-iac@v0.7.0/pkg/scanners/terraform/scanner.go:226 +0x5d0
github.com/aquasecurity/trivy-iac/pkg/scanners/terraform.(*Scanner).ScanFS(0x0?, {0x9b30ee8?, 0xc000b95b20?}, {0x9a70260?, 0xc003527f68?}, {0x9a58958?, 0x2a?})
        /home/runner/go/pkg/mod/github.com/aquasecurity/trivy-iac@v0.7.0/pkg/scanners/terraform/scanner.go:143 +0x46
github.com/aquasecurity/trivy/pkg/misconf.(*Scanner).Scan(0xc00244a6f0, {0x9b30ee8, 0xc000b95b20}, {0x9a70260?, 0xc003527f50?})
        /home/runner/work/trivy/trivy/pkg/misconf/scanner.go:147 +0x190
github.com/aquasecurity/trivy/pkg/fanal/analyzer/config.(*Analyzer).PostAnalyze(0xc000a7afa0, {0x9b30ee8?, 0xc000b95b20?}, {{0x9a70260?, 0xc003527f50?}, {0x9?, 0x0?}})
        /home/runner/work/trivy/trivy/pkg/fanal/analyzer/config/config.go:45 +0x46
github.com/aquasecurity/trivy/pkg/fanal/analyzer.AnalyzerGroup.PostAnalyze({{0xc002dddf80, 0x3, 0x4}, {0xc000967480, 0x7, 0x8}, 0xc002463950}, {0x9b30ee8, 0xc000b95b20}, 0xc000b36910, ...)
        /home/runner/work/trivy/trivy/pkg/fanal/analyzer/analyzer.go:491 +0x2ad
github.com/aquasecurity/trivy/pkg/fanal/artifact/local.Artifact.Inspect({{0x7ffcfe1cd742, 0x1}, {0x7f6ce58d5bd0, 0xc0034bbeb0}, {{{0x0, 0x0, 0x0}, {0xc003394f40, 0x3, 0x4}}, ...}, ...}, ...)
        /home/runner/work/trivy/trivy/pkg/fanal/artifact/local/fs.go:163 +0x48f
github.com/aquasecurity/trivy/pkg/scanner.Scanner.ScanArtifact({{_, _}, {_, _}}, {_, _}, {{0x0, 0x0, 0x0}, {0xc0034bbde0, ...}, ...})
        /home/runner/work/trivy/trivy/pkg/scanner/scan.go:146 +0x103
github.com/aquasecurity/trivy/pkg/commands/artifact.scan({_, _}, {{{0x84d167a, 0xa}, 0x0, 0x0, 0x1, 0x0, 0x45d964b800, {0xc0035ba9a0, ...}, ...}, ...}, ...)
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:704 +0x3b7
github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).scanArtifact(_, {_, _}, {{{0x84d167a, 0xa}, 0x0, 0x0, 0x1, 0x0, 0x45d964b800, ...}, ...}, ...)
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:267 +0xba
github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).scanFS(_, {_, _}, {{{0x84d167a, 0xa}, 0x0, 0x0, 0x1, 0x0, 0x45d964b800, ...}, ...})
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:215 +0xc5
github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).ScanFilesystem(_, {_, _}, {{{0x84d167a, 0xa}, 0x0, 0x0, 0x1, 0x0, 0x45d964b800, ...}, ...})
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:195 +0x211
github.com/aquasecurity/trivy/pkg/commands/artifact.Run({_, _}, {{{0x84d167a, 0xa}, 0x0, 0x0, 0x1, 0x0, 0x45d964b800, {0xc0035ba9a0, ...}, ...}, ...}, ...)
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:428 +0x4ff
github.com/aquasecurity/trivy/pkg/commands.NewConfigCommand.func2(0xc001134c00, {0xc002e44ae0, 0x1, 0x2})
        /home/runner/work/trivy/trivy/pkg/commands/app.go:683 +0x2f2
github.com/spf13/cobra.(*Command).execute(0xc001134c00, {0xc002e44ac0, 0x2, 0x2})
        /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x87c
github.com/spf13/cobra.(*Command).ExecuteC(0xc00023ec00)
        /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5
github.com/spf13/cobra.(*Command).Execute(0x8537fe3?)
        /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992 +0x13
main.run()
        /home/runner/work/trivy/trivy/cmd/trivy/main.go:35 +0x198
main.main()
        /home/runner/work/trivy/trivy/cmd/trivy/main.go:17 +0x13

Operating System

Debian 11 bullseye

Version

$ trivy --version
Version: 0.48.0
Vulnerability DB:
  Version: 2
  UpdatedAt: 2023-10-06 06:18:44.111819375 +0000 UTC
  NextUpdate: 2023-10-06 12:18:44.111818675 +0000 UTC
  DownloadedAt: 2023-10-06 07:59:34.612343851 +0000 UTC
Java DB:
  Version: 1
  UpdatedAt: 2023-10-06 01:07:00.731920298 +0000 UTC
  NextUpdate: 2023-10-09 01:07:00.731919998 +0000 UTC
  DownloadedAt: 2023-10-06 08:00:29.101701499 +0000 UTC
Policy Bundle:
  Digest: sha256:1df8ade71efc830877ca3b1130f83e0c6368e3a45b0d4c0f0418955501644054
  DownloadedAt: 2023-12-06 13:15:18.777009384 +0000 UTC

Checklist

Run trivy image --reset
Read the troubleshooting

The text was updated successfully, but these errors were encountered:

simar7 · 2023-12-06T21:26:28Z

@knqyf263 we'd like to have this in the next patch release (v0.48.1). When would that be?

JAORMX · 2023-12-15T08:51:59Z

We're struggling with the same issue.

knqyf263 · 2023-12-17T09:21:35Z

@simar7 created https://github.com/aquasecurity/trivy/milestone/31

knqyf263 · 2023-12-17T10:45:00Z

@nikpivkin Simar will be on vacation this week. I think we'll cut v0.48.1 and include this fix in v0.49.0. What do you think?

nikpivkin · 2023-12-18T04:53:07Z

@knqyf263 The PR with the fix has been merged, but there has not yet been a trivy-iac release. This critical bug, can it wait until v0.49?

knqyf263 · 2023-12-18T06:27:35Z

Can we release a new version of trivy-iac now?

simar7 · 2023-12-18T06:53:42Z

Can we release a new version of trivy-iac now?

Just did https://github.com/aquasecurity/trivy-iac/releases/tag/v0.7.1

knqyf263 · 2023-12-18T09:20:48Z

@nikpivkin Can you please open a PR to bump trivy-iac to v0.7.1?

nikpivkin · 2023-12-18T09:24:40Z

@knqyf263 Yeah, I'm on it right now

JAORMX · 2023-12-18T09:44:56Z

Could there be a minor release for trivy-action to include these fixes?

commodore-84 · 2023-12-18T10:04:01Z

Not just trivy actions - trivy itself. Could it be updated to 0.49? Currently all pipelines are still failing with [panic ](panic: inconsistent map element types) | cheers

knqyf263 · 2023-12-18T15:52:28Z

Fixed in v0.48.1

nikpivkin added kind/bug Categorizes issue or PR as related to a bug. scan/misconfiguration Issues relating to misconfiguration scanning labels Dec 6, 2023

simar7 assigned simar7 and nikpivkin Dec 6, 2023

simar7 added this to Trivy Roadmap Dec 6, 2023

This was referenced Dec 7, 2023

feat(terraform): fix panic when the for-each arg cannot convert to a map aquasecurity/trivy-iac#60

Merged

feat(terraform): add a method to replace the value in the context aquasecurity/defsec#1504

Merged

ramirezj mentioned this issue Dec 16, 2023

chore(deps): update pre-commit hook antonbabenko/pre-commit-terraform to v1.86.0 grendel-consulting/codebases#105

Merged

1 task

nikpivkin mentioned this issue Dec 18, 2023

chore(deps): bump trivy-iac to v0.7.1 #5797

Merged

6 tasks

knqyf263 closed this as completed Dec 18, 2023

knqyf263 added this to the v0.48.1 milestone Dec 18, 2023

nikpivkin mentioned this issue Jan 19, 2024

feat(terraform): support for random provider #5357

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(terraform): panic when `for-each` contains an object with different attr types #5752

fix(terraform): panic when `for-each` contains an object with different attr types #5752

nikpivkin commented Dec 6, 2023

Description

Desired Behavior

Actual Behavior

Reproduction Steps

Operating System

Version

Checklist

simar7 commented Dec 6, 2023

JAORMX commented Dec 15, 2023

knqyf263 commented Dec 17, 2023

knqyf263 commented Dec 17, 2023

nikpivkin commented Dec 18, 2023 •

edited

Loading

knqyf263 commented Dec 18, 2023

simar7 commented Dec 18, 2023

knqyf263 commented Dec 18, 2023

nikpivkin commented Dec 18, 2023

JAORMX commented Dec 18, 2023

commodore-84 commented Dec 18, 2023

knqyf263 commented Dec 18, 2023 •

edited

Loading

fix(terraform): panic when for-each contains an object with different attr types #5752

fix(terraform): panic when for-each contains an object with different attr types #5752

Comments

nikpivkin commented Dec 6, 2023

Discussed in #5750

Description

Desired Behavior

Actual Behavior

Reproduction Steps

Operating System

Version

Checklist

simar7 commented Dec 6, 2023

JAORMX commented Dec 15, 2023

knqyf263 commented Dec 17, 2023

knqyf263 commented Dec 17, 2023

nikpivkin commented Dec 18, 2023 • edited Loading

knqyf263 commented Dec 18, 2023

simar7 commented Dec 18, 2023

knqyf263 commented Dec 18, 2023

nikpivkin commented Dec 18, 2023

JAORMX commented Dec 18, 2023

commodore-84 commented Dec 18, 2023

knqyf263 commented Dec 18, 2023 • edited Loading

fix(terraform): panic when `for-each` contains an object with different attr types #5752

fix(terraform): panic when `for-each` contains an object with different attr types #5752

nikpivkin commented Dec 18, 2023 •

edited

Loading

knqyf263 commented Dec 18, 2023 •

edited

Loading