-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
skip file / directory patterns or globs #3754
Comments
Hi there, have you looked into trivy.yaml config file -- you could create a config file for each scan e.g. one for config scans that says which files should be excluded & one for fs scans etc. https://aquasecurity.github.io/trivy/v0.38/docs/references/customization/config-file/ trivy/examples/trivy-conf/trivy.yaml Line 11 in 3357ed0
If it is just about ignoring files etc. you could use a .trivyignore file -- tho I think there is a bug so let me follow up on that |
Skipping specific files / dirs is already possible. This is asking about skipping patterns or globs of files or directories, which I don’t think would be possible with the config file suggestion or trivyignore files either? |
Yeah, my bad, I thought trivyignore would make it possible |
This will be less of a problem if scanning initialized |
hi @wyardley you could try the following
// #trivy:ignore:*
module "gke" {
source = "terraform-google-modules/kubernetes-engine/google//modules/beta-private-cluster
Having both of them should not report any errors in remote modules or files found within $ trivy config --skip-dirs=".terraform" .
2023-03-16T00:08:32.496-0700 INFO Misconfiguration scanning is enabled
2023-03-16T00:08:35.497-0700 INFO Detected config files: 2
|
@simar7 Thanks!
So, I don't think this solves the issue, since I believe will also probably ignore any findings from the module itself? The goal should be to not blindly scan all files in I'm not sure if
Based on my understanding of the For example, imagine recursively scanning a structure like this:
where directories may be added or removed over time. It's not practical to ignore each IMO, either trivy needs to be smarter about which stuff it parses and which it doesn't (i.e., don't scan the .terraform directory post initialization, but rather only follow the exact terraform code that's actually being included), and / or to include some way to ignore the contents of a glob of files or directories. Also, I don't think not initializing the code will work either, because then some findings will (in my experience) be missed. Once |
…ptions Addresses: #3754 Signed-off-by: Simar <simar@linux.com>
Thanks understood. I've created a PR to take care of glob support. #3866 |
I can't find a way to write a glob pattern in trivy.yaml that successfully ignores What pattern did you envision using to do this? So far I've tried:
There were probably a few others but none of them work. Trivy still scans into
|
The yaml config and CLI options are equivalent. Can you try to use the CLI options to see if it works in this case? |
@ianchesal fwiw, while I haven't switched over the thing that initially prompted me to ask for this feature yet, I did a quick test and the feature seems to work for me:
Using debug mode, I can confirm that given a simple structure like
trivy will ignore the
That is, if you have
there doesn't seem to be an obvious glob pattern that will match
@simar7 is there a way to get the expected type of globbing behavior (if so, I'd be happy to make a PR to enhance the docs) |
@simar7 I have the same output when using the --skip-dirs flag cli or when using the configuration file using the --skip-dirs flag: trivy config -d . --skip-dirs ".terraform/**"
2023-07-21T09:48:38.988+0600 INFO Loaded trivy.yaml
2023-07-21T09:48:38.989+0600 DEBUG Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
2023-07-21T09:48:38.999+0600 DEBUG cache dir: /Users/tososomaru/Library/Caches/trivy
2023-07-21T09:48:38.999+0600 DEBUG Module dir: /Users/tososomaru/.trivy/modules
2023-07-21T09:48:38.999+0600 INFO Misconfiguration scanning is enabled
2023-07-21T09:48:39.000+0600 DEBUG Policies successfully loaded from disk
2023-07-21T09:48:39.026+0600 DEBUG Walk the file tree rooted at '.' in parallel
2023-07-21T09:48:39.026+0600 DEBUG Skipping directory: .terraform/providers
2023-07-21T09:48:39.026+0600 DEBUG Skipping directory: .terraform/modules
2023-07-21T09:48:39.026+0600 DEBUG Scanning Helm files for misconfigurations...
2023-07-21T09:48:39.026+0600 DEBUG Scanning Terraform files for misconfigurations...
2023-07-21T09:48:42.835+0600 DEBUG OS is not detected.
2023-07-21T09:48:42.835+0600 INFO Detected config files: 3
2023-07-21T09:48:42.835+0600 DEBUG Scanned config file: .
2023-07-21T09:48:42.835+0600 DEBUG Scanned config file: git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/sa.tf
2023-07-21T09:48:42.835+0600 DEBUG Scanned config file: git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf
git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf (terraform)
Tests: 18 (SUCCESSES: 16, FAILURES: 2, EXCEPTIONS: 0)
Failures: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 1, CRITICAL: 0)
MEDIUM: Cluster does not have a network policy enabled.
═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Enabling a network policy allows the segregation of network traffic by namespace
See https://avd.aquasec.com/misconfig/avd-gcp-0056
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf:38
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
22 resource "google_container_cluster" "primary" {
..
38 [ enabled = network_policy.value.enabled
...
441 }
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
HIGH: Cluster does not have master authorized networks enabled.
═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Enabling authorized networks means you can restrict master access to a fixed set of CIDR ranges
See https://avd.aquasec.com/misconfig/avd-gcp-0061
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf:22-441
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
22 ┌ resource "google_container_cluster" "primary" {
23 │ provider = google-beta
24 │
25 │ name = var.name
26 │ description = var.description
27 │ project = var.project_id
28 │ resource_labels = var.cluster_resource_labels
29 │
30 └ location = local.location
..
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
using the configuration file: scan:
skip-dirs:
- .terraform/** trivy config -d . --config trivy.yaml
2023-07-21T09:49:07.860+0600 INFO Loaded trivy.yaml
2023-07-21T09:49:07.861+0600 DEBUG Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
2023-07-21T09:49:07.870+0600 DEBUG cache dir: /Users/tososomaru/Library/Caches/trivy
2023-07-21T09:49:07.870+0600 DEBUG Module dir: /Users/tososomaru/.trivy/modules
2023-07-21T09:49:07.870+0600 INFO Misconfiguration scanning is enabled
2023-07-21T09:49:07.870+0600 DEBUG Policies successfully loaded from disk
2023-07-21T09:49:07.891+0600 DEBUG Walk the file tree rooted at '.' in parallel
2023-07-21T09:49:07.892+0600 DEBUG Skipping directory: .terraform/providers
2023-07-21T09:49:07.892+0600 DEBUG Skipping directory: .terraform/modules
2023-07-21T09:49:07.892+0600 DEBUG Scanning Helm files for misconfigurations...
2023-07-21T09:49:07.892+0600 DEBUG Scanning Terraform files for misconfigurations...
2023-07-21T09:49:11.355+0600 DEBUG OS is not detected.
2023-07-21T09:49:11.355+0600 INFO Detected config files: 3
2023-07-21T09:49:11.355+0600 DEBUG Scanned config file: .
2023-07-21T09:49:11.355+0600 DEBUG Scanned config file: git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf
2023-07-21T09:49:11.355+0600 DEBUG Scanned config file: git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/sa.tf
git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf (terraform)
Tests: 18 (SUCCESSES: 16, FAILURES: 2, EXCEPTIONS: 0)
Failures: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 1, CRITICAL: 0)
MEDIUM: Cluster does not have a network policy enabled.
═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Enabling a network policy allows the segregation of network traffic by namespace
See https://avd.aquasec.com/misconfig/avd-gcp-0056
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf:38
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
22 resource "google_container_cluster" "primary" {
..
38 [ enabled = network_policy.value.enabled
...
441 }
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
HIGH: Cluster does not have master authorized networks enabled.
═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Enabling authorized networks means you can restrict master access to a fixed set of CIDR ranges
See https://avd.aquasec.com/misconfig/avd-gcp-0061
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf:22-441
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
22 ┌ resource "google_container_cluster" "primary" {
23 │ provider = google-beta
24 │
25 │ name = var.name
26 │ description = var.description
27 │ project = var.project_id
28 │ resource_labels = var.cluster_resource_labels
29 │
30 └ location = local.location
..
────────────────────────────────────────────────────────────────────────── |
Go's
Looks like https://github.com/ganbarodigital/go_glob doesn't support extended globbing either. golang/go#11862 has some discussion about it. Based on the suggestion there, something like the below could work:
|
A gitignore type syntax would also work quite well — some existing golang interpretations, e.g., https://pkg.go.dev/github.com/sabhiram/go-gitignore |
@ianchesal at the moment, you can use the scan:
skip-files:
- "**/.terraform/**/*" |
@nikpivkin ah! Thank you! That seems to do it. For certain, the skip-dirs globbing doesn't seem to have a way to accomplish this. As @wyardley notes, it doesn't seem to match If I can help with a patch here I'm more than happy to pitch in. Thanks all for the work on this! |
@ianchesal thanks, I opened PR with fixes |
* docs(cli): update help string for file and dir skipping - Update the contextual help messages - Add some additional examples (and clarify YAML file configuration) for globbing - Update docs - Fix broken link in skipping docs See also #3754 Signed-off-by: William Yardley <wyardley@users.noreply.github.com> * docs: revert --------- Signed-off-by: William Yardley <wyardley@users.noreply.github.com> Co-authored-by: knqyf263 <knqyf263@gmail.com>
…#4872) * docs(cli): update help string for file and dir skipping - Update the contextual help messages - Add some additional examples (and clarify YAML file configuration) for globbing - Update docs - Fix broken link in skipping docs See also aquasecurity#3754 Signed-off-by: William Yardley <wyardley@users.noreply.github.com> * docs: revert --------- Signed-off-by: William Yardley <wyardley@users.noreply.github.com> Co-authored-by: knqyf263 <knqyf263@gmail.com>
It would be useful to be able to skip directories or files using globs or simple regex type patterns.
Use case: Using trivy to scan initialized Terraform config directories with modules that have embedded examples can take a long time to scan, and turn up false positives
For example, if a module like https://github.com/terraform-google-modules/terraform-google-kubernetes-engine is included
Presumably, while
trivy
should be smart enough to follow actual module code that's used, it shouldn't naively scan all tf files in the.terraform
directory of an initialized repo. (side note: if you believe this behavior is a bug, I can also open a bug report for this separately).While the
--skip-dirs
option exists, it doesn't seem to support ignoring a glob pattern or regex, so you can't do, e.g.,--skip-dirs **/.terraform
or--skip-files **/.terraform/**/*.tf
simple repro:
The text was updated successfully, but these errors were encountered: