fix(misconf): directory filtering after scanning

[nikpivkin]

Trivy supports scanning Terraform modules that are outside the scan directory, but they cannot be skipped using the -skip-dirs, --skip-files flags. We need to filter the result based on the directories after scanning.

Discussed in #7191

^{Originally posted by MatthiasScholzTW July 19, 2024}

Description

When using a subdirectory for the scanning the commands --skip-dirs and --skip-files are ignored.

Example:

• trivy fs --scanners misconfig --skip-dirs "../modules" deployments

Desired Behavior

The skipping functionality supports using path within a project root folder.

Actual Behavior

The expressions provided within --skip-dirs and --skip-files are ignored.

Reproduction Steps

A reproduction sample can be found as a [repository here](https://github.com/MatthiasScholzTW/bug_trivy_skip.git).

General steps to reproduce:
1. Create terraform module with a resource with a misconfiguration
2. Reference the module from another folder within the repository
3. Run `trivy fs --scanners misconfig --skip-dirs "modules" .` -> no issues reported (expected)
4. Run `trivy fs --scanners misconfig --skip-dirs "../modules" deployments -> issue reported (not expected)

Target

Filesystem

Scanner

Misconfiguration

Output Format

None

Mode

Standalone

Debug Output

2024-07-19T09:35:30+02:00       DEBUG   Cache dir       dir="/Users/matthias/Library/Caches/trivy"
2024-07-19T09:35:30+02:00       DEBUG   Parsed severities       severities=[UNKNOWN LOW MEDIUM HIGH CRITICAL]
2024-07-19T09:35:30+02:00       DEBUG   Ignore statuses statuses=[]
2024-07-19T09:35:30+02:00       INFO    Misconfiguration scanning is enabled
2024-07-19T09:35:30+02:00       DEBUG   Policies successfully loaded from disk
2024-07-19T09:35:30+02:00       DEBUG   Enabling misconfiguration scanners      scanners=[azure-arm cloudformation dockerfile helm kubernetes terraform terraformplan-json terraformplan-snapshot]
2024-07-19T09:35:30+02:00       DEBUG   Initializing scan cache...      type="memory"
2024-07-19T09:35:30+02:00       DEBUG   [nuget] The nuget packages directory couldn't be found. License search disabled
2024-07-19T09:35:30+02:00       DEBUG   Scanning files for misconfigurations... scanner="Terraform"
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.668798000 terraform.scanner                Scanning [&{%!s(*mapfs.file=&{ [] {. 256 2147484096 {13951768674766166528 270705001 0x10cf17c60} <nil>} {{{0 0} {[] {} 0x140032202b0} map[mycode.tf:0x140017c19f0] 0}}}) deployments}] at '.'...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.670569000 terraform.scanner.rego           Overriding filesystem for checks!
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.671495000 terraform.scanner.rego           Loaded 3 embedded libraries.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.696682000 terraform.scanner.rego           Loaded 192 embedded policies.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.733942000 terraform.scanner.rego           Loaded 195 checks from disk.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.734160000 terraform.scanner.rego           Overriding filesystem for data!
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.901763000 terraform.parser.<root>          Setting project/module root to '.'
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.901798000 terraform.parser.<root>          Parsing FS from '.'
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.901824000 terraform.parser.<root>          Parsing 'mycode.tf'...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.903541000 terraform.parser.<root>          Added file mycode.tf.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.903795000 terraform.scanner                Scanning root module '.'...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.903800000 terraform.parser.<root>          Setting project/module root to '.'
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.903803000 terraform.parser.<root>          Parsing FS from '.'
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.903813000 terraform.parser.<root>          Parsing 'mycode.tf'...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.903859000 terraform.parser.<root>          Added file mycode.tf.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.903869000 terraform.parser.<root>          Evaluating module...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.903884000 terraform.parser.<root>          Read 1 block(s) and 0 ignore(s) for module 'root' (1 file[s])...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.903908000 terraform.parser.<root>          Added 0 variables from tfvars.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.903980000 terraform.parser.<root>          Working directory for module evaluation is "/Users/demo/bug_trivy"
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904046000 terraform.parser.<root>.evaluator Filesystem key is '975327516ac7bb24384705e60a69d80c25f655b086befe687b2866178a33c894'
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904049000 terraform.parser.<root>.evaluator Starting module evaluation...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904069000 terraform.parser.<root>.evaluator Starting submodule evaluation...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904072000 terraform.parser.<root>.evaluator locating non-initialized module '../modules'...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904075000 terraform.parser.<root>.evaluator.resolver Resolving module 'module.use_bad_configuration' with source: '../modules'...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904081000 terraform.parser.<root>.evaluator.resolver Module 'module.use_bad_configuration' resolved locally to ../modules
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904269000 terraform.parser.<root>.evaluator.resolver Module path is ../modules
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904273000 terraform.parser.<root>.evaluator Module 'module.use_bad_configuration' resolved to path '../modules' in filesystem '&{%!s(*mapfs.file=&{ [] {. 256 2147484096 {13951768674766166528 270705001 0x10cf17c60} <nil>} {{{0 0} {[] {} 0x140032203e0} map[] 0}}}) deployments}' with prefix ''
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904275000 terraform.parser.<use_bad_configuration> Parsing FS from '../modules'
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904335000 terraform.parser.<use_bad_configuration> Parsing '../modules/misconfiguration.tf'...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904616000 terraform.parser.<use_bad_configuration> Added file ../modules/misconfiguration.tf.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904628000 terraform.parser.<root>.evaluator Loaded module "use_bad_configuration" from "../modules".
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904631000 terraform.parser.<use_bad_configuration> Evaluating module...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904660000 terraform.parser.<use_bad_configuration> Read 2 block(s) and 0 ignore(s) for module 'use_bad_configuration' (1 file[s])...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904665000 terraform.parser.<use_bad_configuration> Added 2 input variables from module definition.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904673000 terraform.parser.<use_bad_configuration> Working directory for module evaluation is "/Users/demo/bug_trivy"
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904687000 terraform.parser.<root>.evaluator Evaluating submodule use_bad_configuration
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904861000 terraform.parser.<use_bad_configuration>.evaluator Filesystem key is '975327516ac7bb24384705e60a69d80c25f655b086befe687b2866178a33c894'
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904864000 terraform.parser.<use_bad_configuration>.evaluator Starting module evaluation...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904947000 terraform.parser.<use_bad_configuration>.evaluator Starting submodule evaluation...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904949000 terraform.parser.<use_bad_configuration>.evaluator All submodules are evaluated at i=0
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904954000 terraform.parser.<use_bad_configuration>.evaluator Starting post-submodule evaluation...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.904985000 terraform.parser.<use_bad_configuration>.evaluator Finished processing 0 submodule(s).
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.905145000 terraform.parser.<use_bad_configuration>.evaluator Module evaluation complete.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.905165000 terraform.parser.<root>.evaluator Submodule use_bad_configuration inputs unchanged
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.905167000 terraform.parser.<root>.evaluator All submodules are evaluated at i=1
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.905169000 terraform.parser.<root>.evaluator Starting post-submodule evaluation...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.905184000 terraform.parser.<root>.evaluator Finished processing 1 submodule(s).
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.905424000 terraform.parser.<root>.evaluator Module evaluation complete.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.905426000 terraform.parser.<root>          Finished parsing module 'root'.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.905429000 terraform.executor               Adapting modules...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.907104000 terraform.executor               Adapted 2 module(s) into defsec state data.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.907107000 terraform.executor               Using max routines of 13
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.907183000 terraform.executor               Initialized 487 rule(s).
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.907186000 terraform.executor               Created pool with 13 worker(s) to apply rules.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.907648000 terraform.scanner.rego           Scanning 1 inputs...
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.909260000 terraform.executor               Finished applying rules.
2024-07-19T09:35:30+02:00       DEBUG   [misconf] 35:30.909281000 terraform.executor               Applying ignores...
2024-07-19T09:35:30+02:00       DEBUG   OS is not detected.
2024-07-19T09:35:30+02:00       INFO    Detected config files   num=2
2024-07-19T09:35:30+02:00       DEBUG   Scanned config file     path="."
2024-07-19T09:35:30+02:00       DEBUG   Scanned config file     path="../modules/misconfiguration.tf"

Operating System

macOS 14.5

Version

Version: 0.53.0
Vulnerability DB:
  Version: 2
  UpdatedAt: 2024-07-19 06:11:22.340274454 +0000 UTC
  NextUpdate: 2024-07-19 12:11:22.340274304 +0000 UTC
  DownloadedAt: 2024-07-19 06:19:54.571889 +0000 UTC
Check Bundle:
  Digest: sha256:ef2d9ad4fce0f933b20a662004d7e55bf200987c180e7f2cd531af631f408bb3
  DownloadedAt: 2024-07-19 06:13:50.914793 +0000 UTC

Checklist

• Run trivy clean --all
• Read the troubleshooting

[nikpivkin]

@knqyf263 Should we apply re-filtering only to misconfiguration results?