Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(misconf): Add support for --cf-params for CFT #5507

Merged
merged 33 commits into from
Nov 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
c4021ff
refactor(misconf): Update refactored dependencies
simar7 Sep 25, 2023
c74f3ac
update deps
simar7 Sep 25, 2023
88b78fa
update go mod
simar7 Sep 25, 2023
1ff26d2
update trivy-* deps
simar7 Sep 27, 2023
a96a3b6
update policy bundle path
simar7 Sep 27, 2023
b86d14e
bump defsec
simar7 Oct 11, 2023
d9342b2
update go mod
simar7 Oct 12, 2023
97b85f5
update go mod
simar7 Oct 16, 2023
98864a7
Merge branch 'main' into defsec-refactor-update
simar7 Oct 16, 2023
be682e8
go mod tidy
simar7 Oct 28, 2023
e0b3e1b
Merge branch 'main' into defsec-refactor-update
simar7 Oct 30, 2023
dc0839a
go mod tidy
simar7 Oct 30, 2023
bfdac0b
update docs
simar7 Oct 30, 2023
6f45fae
fix trivy-iac import
simar7 Oct 31, 2023
569b141
update trivy-iac to v0.5.2
simar7 Nov 1, 2023
91f949b
Merge branch 'main' into defsec-refactor-update
simar7 Nov 1, 2023
09d1775
Merge branch 'main' into defsec-refactor-update
simar7 Nov 2, 2023
e2bb9fc
pin testcontainers@v0.23.0
simar7 Nov 2, 2023
998ca85
feat(misconf): Add support for `--cf-params` for CloudFormation param…
simar7 Nov 1, 2023
9165d11
update docs
simar7 Nov 1, 2023
2ca15d9
pin testcontainers@v0.23.0
simar7 Nov 2, 2023
e59f91d
update docs
simar7 Nov 2, 2023
73de07f
go mod update
simar7 Nov 4, 2023
689f60a
rename vars
simar7 Nov 4, 2023
8b6f7a3
go mod tidy
simar7 Nov 7, 2023
8d27d95
use .WithParameterFiles()
simar7 Nov 7, 2023
62dace8
Merge branch 'main' into support-cf-params-2
simar7 Nov 10, 2023
349d3a9
go mod tidy
simar7 Nov 10, 2023
05b9d9a
refactor: use a dedicated FS for tf and CF configs
nikpivkin Nov 14, 2023
cb21634
Merge branch 'main' into support-cf-params-2
nikpivkin Nov 14, 2023
a9275bf
fix docs
simar7 Nov 15, 2023
8ae3aca
fix usage help
simar7 Nov 15, 2023
8d8d834
disable cf/tf flag where not needed
simar7 Nov 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions docs/docs/coverage/iac/cloudformation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,34 @@
Trivy supports the scanners listed in the table below.

| Scanner | Supported |
| :----------------: | :-------: |
|:------------------:|:---------:|
| [Misconfiguration] | ✓ |
| [Secret] | ✓ |

It supports the following formats.

| Format | Supported |
| :----: | :-------: |
|:------:|:---------:|
| JSON | ✓ |
| YAML | ✓ |

## Misconfiguration
Trivy recursively searches directories and scans all found CloudFormation files.
It evaluates properties, functions, and other elements within CloudFormation files to detect misconfigurations.

### Value Overrides
nikpivkin marked this conversation as resolved.
Show resolved Hide resolved
You can provide `cf-params` with path to [CloudFormation Parameters] file to Trivy to scan your CloudFormation code with parameters.

```bash
trivy conf --cf-params params.json ./infrastructure/cf
```

You can check a [CloudFormation Parameters Example]

## Secret
The secret scan is performed on plain text files, with no special treatment for CloudFormation.

[Misconfiguration]: ../../scanner/misconfiguration/index.md
[Secret]: ../../scanner/secret.md
[Secret]: ../../scanner/secret.md
[CloudFormation Parameters]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html
[CloudFormation Parameters Example]: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudformation/deploy.html#supported-json-syntax
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ trivy aws [flags]
```
--account string The AWS account to scan. It's useful to specify this when reviewing cached results for multiple accounts.
--arn string The AWS ARN to show results for. Useful to filter results once a scan is cached.
--cf-params strings specify paths to override the CloudFormation parameters files
--compliance string compliance report to generate (aws-cis-1.2,aws-cis-1.4)
--config-data strings specify paths from which data for the Rego policies will be recursively loaded
--config-policy strings specify the paths to the Rego policy files or to the directories containing them, applying config files
Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ trivy config [flags] DIR
```
--cache-backend string cache backend (e.g. redis://localhost:6379) (default "fs")
--cache-ttl duration cache TTL when using redis as cache backend
--cf-params strings specify paths to override the CloudFormation parameters files
--clear-cache clear image caches without scanning
--compliance string compliance report to generate
--config-data strings specify paths from which data for the Rego policies will be recursively loaded
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ trivy filesystem [flags] PATH
```
--cache-backend string cache backend (e.g. redis://localhost:6379) (default "fs")
--cache-ttl duration cache TTL when using redis as cache backend
--cf-params strings specify paths to override the CloudFormation parameters files
--clear-cache clear image caches without scanning
--compliance string compliance report to generate
--config-data strings specify paths from which data for the Rego policies will be recursively loaded
Expand Down
1 change: 0 additions & 1 deletion docs/docs/references/configuration/cli/trivy_image.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,6 @@ trivy image [flags] IMAGE_NAME
--slow scan over time with lower CPU and memory utilization
-t, --template string output template
--tf-exclude-downloaded-modules exclude misconfigurations for downloaded terraform modules
--tf-vars strings specify paths to override the Terraform tfvars files
--token string for authentication in client/server mode
--token-header string specify a header name for token in client/server mode (default "Trivy-Token")
--trace enable more verbose trace output for custom queries
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,6 @@ trivy kubernetes [flags] { cluster | all | specific resources like kubectl. eg:
--slow scan over time with lower CPU and memory utilization
-t, --template string output template
--tf-exclude-downloaded-modules exclude misconfigurations for downloaded terraform modules
--tf-vars strings specify paths to override the Terraform tfvars files
--tolerations strings specify node-collector job tolerations (example: key1=value1:NoExecute,key2=value2:NoSchedule)
--trace enable more verbose trace output for custom queries
--username strings username. Comma-separated usernames allowed.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ trivy repository [flags] (REPO_PATH | REPO_URL)
--branch string pass the branch name to be scanned
--cache-backend string cache backend (e.g. redis://localhost:6379) (default "fs")
--cache-ttl duration cache TTL when using redis as cache backend
--cf-params strings specify paths to override the CloudFormation parameters files
--clear-cache clear image caches without scanning
--commit string pass the commit hash to be scanned
--config-data strings specify paths from which data for the Rego policies will be recursively loaded
Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_rootfs.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ trivy rootfs [flags] ROOTDIR
```
--cache-backend string cache backend (e.g. redis://localhost:6379) (default "fs")
--cache-ttl duration cache TTL when using redis as cache backend
--cf-params strings specify paths to override the CloudFormation parameters files
--clear-cache clear image caches without scanning
--config-data strings specify paths from which data for the Rego policies will be recursively loaded
--config-policy strings specify the paths to the Rego policy files or to the directories containing them, applying config files
Expand Down
1 change: 0 additions & 1 deletion docs/docs/references/configuration/cli/trivy_vm.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ trivy vm [flags] VM_IMAGE
--slow scan over time with lower CPU and memory utilization
-t, --template string output template
--tf-exclude-downloaded-modules exclude misconfigurations for downloaded terraform modules
--tf-vars strings specify paths to override the Terraform tfvars files
--token string for authentication in client/server mode
--token-header string specify a header name for token in client/server mode (default "Trivy-Token")
--vuln-type strings comma-separated list of vulnerability types (os,library) (default [os,library])
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ require (
github.com/aquasecurity/trivy-iac v0.5.2
github.com/aquasecurity/trivy-java-db v0.0.0-20230209231723-7cddb1406728
github.com/aquasecurity/trivy-kubernetes v0.5.9-0.20231023143623-130fa1a0e44a
github.com/aquasecurity/trivy-policies v0.4.0
github.com/aquasecurity/trivy-policies v0.5.0
github.com/aws/aws-sdk-go-v2 v1.22.1
github.com/aws/aws-sdk-go-v2/config v1.18.45
github.com/aws/aws-sdk-go-v2/credentials v1.13.43
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -352,8 +352,8 @@ github.com/aquasecurity/trivy-java-db v0.0.0-20230209231723-7cddb1406728 h1:0eS+
github.com/aquasecurity/trivy-java-db v0.0.0-20230209231723-7cddb1406728/go.mod h1:Ldya37FLi0e/5Cjq2T5Bty7cFkzUDwTcPeQua+2M8i8=
github.com/aquasecurity/trivy-kubernetes v0.5.9-0.20231023143623-130fa1a0e44a h1:x1Z+k7snLeDjTLnvTd4UPNo0HPMP6SNWxZTVU5smuA0=
github.com/aquasecurity/trivy-kubernetes v0.5.9-0.20231023143623-130fa1a0e44a/go.mod h1:e3iMAeJ1V/iPsNcRBVd9aDqj1YAXeURzv3qLurbloUk=
github.com/aquasecurity/trivy-policies v0.4.0 h1:20dEiWkiTrFXAiPLVshfJ/x1i9LvDDsBJ/CAb4fNEBI=
github.com/aquasecurity/trivy-policies v0.4.0/go.mod h1:Wqj81EIp4lDQGVzbPalKLNucR7c96YLQbfdA60KpEkQ=
github.com/aquasecurity/trivy-policies v0.5.0 h1:7GukJhiEQpKg8VQH3PkwZOyFqO0J6hGmUbt7jne5mhU=
github.com/aquasecurity/trivy-policies v0.5.0/go.mod h1:YPefENNCAcbPxMDgKBWxjLmhyzYnlAY/HIH89VFaogY=
github.com/arbovm/levenshtein v0.0.0-20160628152529-48b4e1c0c4d0 h1:jfIu9sQUG6Ig+0+Ap1h4unLjW6YQJpKZVmUzxsD4E/Q=
github.com/arbovm/levenshtein v0.0.0-20160628152529-48b4e1c0c4d0/go.mod h1:t2tdKJDJF9BV14lnkjHmOQgcvEKgtqs5a1N3LNdJhGE=
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
Expand Down
18 changes: 14 additions & 4 deletions pkg/commands/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -232,12 +232,16 @@ func NewImageCommand(globalFlags *flag.GlobalFlagGroup) *cobra.Command {
compliance.Values = []string{types.ComplianceDockerCIS}
reportFlagGroup.Compliance = &compliance // override usage as the accepted values differ for each subcommand.

misconfFlagGroup := flag.NewMisconfFlagGroup()
misconfFlagGroup.CloudformationParamVars = nil // disable '--cf-params'
misconfFlagGroup.TerraformTFVars = nil // disable '--tf-vars'

imageFlags := &flag.Flags{
CacheFlagGroup: flag.NewCacheFlagGroup(),
DBFlagGroup: flag.NewDBFlagGroup(),
ImageFlagGroup: flag.NewImageFlagGroup(), // container image specific
LicenseFlagGroup: flag.NewLicenseFlagGroup(),
MisconfFlagGroup: flag.NewMisconfFlagGroup(),
MisconfFlagGroup: misconfFlagGroup,
ModuleFlagGroup: flag.NewModuleFlagGroup(),
RemoteFlagGroup: flag.NewClientFlags(), // for client/server mode
RegistryFlagGroup: flag.NewRegistryFlagGroup(),
Expand Down Expand Up @@ -896,12 +900,16 @@ func NewKubernetesCommand(globalFlags *flag.GlobalFlagGroup) *cobra.Command {
})
reportFlagGroup.Format = &formatFlag

misconfFlagGroup := flag.NewMisconfFlagGroup()
misconfFlagGroup.CloudformationParamVars = nil // disable '--cf-params'
misconfFlagGroup.TerraformTFVars = nil // disable '--tf-vars'

k8sFlags := &flag.Flags{
CacheFlagGroup: flag.NewCacheFlagGroup(),
DBFlagGroup: flag.NewDBFlagGroup(),
ImageFlagGroup: imageFlags,
K8sFlagGroup: flag.NewK8sFlagGroup(), // kubernetes-specific flags
MisconfFlagGroup: flag.NewMisconfFlagGroup(),
MisconfFlagGroup: misconfFlagGroup,
RegoFlagGroup: flag.NewRegoFlagGroup(),
ReportFlagGroup: reportFlagGroup,
ScanFlagGroup: scanFlags,
Expand Down Expand Up @@ -1046,8 +1054,10 @@ func NewVMCommand(globalFlags *flag.GlobalFlagGroup) *cobra.Command {
},
},
}
vmFlags.ReportFlagGroup.ReportFormat = nil // disable '--report'
vmFlags.ScanFlagGroup.IncludeDevDeps = nil // disable '--include-dev-deps'
vmFlags.ReportFlagGroup.ReportFormat = nil // disable '--report'
vmFlags.ScanFlagGroup.IncludeDevDeps = nil // disable '--include-dev-deps'
vmFlags.MisconfFlagGroup.CloudformationParamVars = nil // disable '--cf-params'
vmFlags.MisconfFlagGroup.TerraformTFVars = nil // disable '--tf-vars'

cmd := &cobra.Command{
Use: "vm [flags] VM_IMAGE",
Expand Down
1 change: 1 addition & 0 deletions pkg/commands/artifact/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -583,6 +583,7 @@ func initScannerConfig(opts flag.Options, cacheClient cache.Cache) (ScannerConfi
HelmFileValues: opts.HelmFileValues,
HelmStringValues: opts.HelmStringValues,
TerraformTFVars: opts.TerraformTFVars,
CloudFormationParamVars: opts.CloudFormationParamVars,
K8sVersion: opts.K8sVersion,
DisableEmbeddedPolicies: disableEmbedded,
DisableEmbeddedLibraries: disableEmbedded,
Expand Down
62 changes: 60 additions & 2 deletions pkg/fanal/artifact/local/fs_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -854,7 +854,7 @@ func TestCloudFormationMisconfigurationScan(t *testing.T) {
Args: cache.ArtifactCachePutBlobArgs{
BlobIDAnything: true,
BlobInfo: types.BlobInfo{
SchemaVersion: 2,
SchemaVersion: types.BlobJSONSchemaVersion,
Misconfigurations: []types.Misconfiguration{
{
FileType: "cloudformation",
Expand Down Expand Up @@ -1014,6 +1014,64 @@ func TestCloudFormationMisconfigurationScan(t *testing.T) {
},
},
},
{
name: "CloudFormation parameters outside the scan directory",
fields: fields{
dir: "./testdata/misconfig/cloudformation/params/code/src",
},
artifactOpt: artifact.Option{
MisconfScannerOption: misconf.ScannerOption{
RegoOnly: true,
Namespaces: []string{"user"},
PolicyPaths: []string{"./testdata/misconfig/cloudformation/params/code/rego"},
CloudFormationParamVars: []string{"./testdata/misconfig/cloudformation/params/cfparams.json"},
DisableEmbeddedPolicies: true,
DisableEmbeddedLibraries: true,
},
},
putBlobExpectation: cache.ArtifactCachePutBlobExpectation{
Args: cache.ArtifactCachePutBlobArgs{
BlobIDAnything: true,
BlobInfo: types.BlobInfo{
SchemaVersion: types.BlobJSONSchemaVersion,
Misconfigurations: []types.Misconfiguration{
{
FileType: "cloudformation",
FilePath: "main.yaml",
Successes: types.MisconfResults{
{
Namespace: "user.something",
Query: "data.user.something.deny",
PolicyMetadata: types.PolicyMetadata{
ID: "TEST001",
AVDID: "AVD-TEST-0001",
Type: "CloudFormation Security Check",
Title: "Bad stuff is bad",
Description: "Its not good!",
Severity: "HIGH",
RecommendedActions: "Remove bad stuff",
},
CauseMetadata: types.CauseMetadata{
Provider: "AWS",
Service: "sqs",
},
},
},
},
},
},
},
Returns: cache.ArtifactCachePutBlobReturns{},
},
want: types.ArtifactReference{
Name: "testdata/misconfig/cloudformation/params/code/src",
Type: types.ArtifactFilesystem,
ID: "sha256:0c66c19a4df3ecc11db9f90fbc921f1050325c05c480847369e07ee309e8a897",
BlobIDs: []string{
"sha256:0c66c19a4df3ecc11db9f90fbc921f1050325c05c480847369e07ee309e8a897",
},
},
},
{
name: "passed",
fields: fields{
Expand All @@ -1032,7 +1090,7 @@ func TestCloudFormationMisconfigurationScan(t *testing.T) {
Args: cache.ArtifactCachePutBlobArgs{
BlobIDAnything: true,
BlobInfo: types.BlobInfo{
SchemaVersion: 2,
SchemaVersion: types.BlobJSONSchemaVersion,
Misconfigurations: []types.Misconfiguration{
{
FileType: "cloudformation",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[
{
"ParameterKey": "KmsMasterKeyId",
"ParameterValue": "some_id"
}
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# METADATA
# title: "Bad stuff is bad"
# description: "Its not good!"
# scope: package
# schemas:
# - input: schema["cloud"]
# custom:
# avd_id: AVD-TEST-0001
# id: TEST001
# provider: aws
# service: sqs
# severity: HIGH
# short_code: foo-bar-baz
# recommended_action: "Remove bad stuff"

package user.something

deny[res] {
qs := input.aws.sqs.queues[_]
qs.encryption.kmskeyid.value == ""
res := "No unencrypted queues allowed!"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
AWSTemplateFormatVersion: 2010-09-09
Parameters:
KmsMasterKeyId:
Type: String
Resources:
TestQueue:
Type: 'AWS::SQS::Queue'
Properties:
QueueName: worst-possible-queue
KmsMasterKeyId: !Ref KmsMasterKeyId
43 changes: 27 additions & 16 deletions pkg/flag/misconf_flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,12 @@ var (
Default: []string{},
Usage: "specify paths to override the Terraform tfvars files",
}
CfParamsFlag = Flag{
Name: "cf-params",
nikpivkin marked this conversation as resolved.
Show resolved Hide resolved
ConfigName: "misconfiguration.cloudformation.params",
Default: []string{},
Usage: "specify paths to override the CloudFormation parameters files",
}
TerraformExcludeDownloaded = Flag{
Name: "tf-exclude-downloaded-modules",
ConfigName: "misconfiguration.terraform.exclude-downloaded-modules",
Expand All @@ -69,7 +75,7 @@ var (
}
)

// MisconfFlagGroup composes common printer flag structs used for commands providing misconfinguration scanning.
// MisconfFlagGroup composes common printer flag structs used for commands providing misconfiguration scanning.
type MisconfFlagGroup struct {
IncludeNonFailures *Flag
ResetPolicyBundle *Flag
Expand All @@ -81,6 +87,7 @@ type MisconfFlagGroup struct {
HelmFileValues *Flag
HelmStringValues *Flag
TerraformTFVars *Flag
CloudformationParamVars *Flag
TerraformExcludeDownloaded *Flag
}

Expand All @@ -90,12 +97,13 @@ type MisconfOptions struct {
PolicyBundleRepository string

// Values Files
HelmValues []string
HelmValueFiles []string
HelmFileValues []string
HelmStringValues []string
TerraformTFVars []string
TfExcludeDownloaded bool
HelmValues []string
HelmValueFiles []string
HelmFileValues []string
HelmStringValues []string
TerraformTFVars []string
CloudFormationParamVars []string
TfExcludeDownloaded bool
}

func NewMisconfFlagGroup() *MisconfFlagGroup {
Expand All @@ -109,6 +117,7 @@ func NewMisconfFlagGroup() *MisconfFlagGroup {
HelmStringValues: &HelmSetStringFlag,
HelmValueFiles: &HelmValuesFileFlag,
TerraformTFVars: &TfVarsFlag,
CloudformationParamVars: &CfParamsFlag,
TerraformExcludeDownloaded: &TerraformExcludeDownloaded,
}
}
Expand All @@ -128,19 +137,21 @@ func (f *MisconfFlagGroup) Flags() []*Flag {
f.HelmStringValues,
f.TerraformTFVars,
f.TerraformExcludeDownloaded,
f.CloudformationParamVars,
}
}

func (f *MisconfFlagGroup) ToOptions() (MisconfOptions, error) {
return MisconfOptions{
IncludeNonFailures: getBool(f.IncludeNonFailures),
ResetPolicyBundle: getBool(f.ResetPolicyBundle),
PolicyBundleRepository: getString(f.PolicyBundleRepository),
HelmValues: getStringSlice(f.HelmValues),
HelmValueFiles: getStringSlice(f.HelmValueFiles),
HelmFileValues: getStringSlice(f.HelmFileValues),
HelmStringValues: getStringSlice(f.HelmStringValues),
TerraformTFVars: getStringSlice(f.TerraformTFVars),
TfExcludeDownloaded: getBool(f.TerraformExcludeDownloaded),
IncludeNonFailures: getBool(f.IncludeNonFailures),
ResetPolicyBundle: getBool(f.ResetPolicyBundle),
PolicyBundleRepository: getString(f.PolicyBundleRepository),
HelmValues: getStringSlice(f.HelmValues),
HelmValueFiles: getStringSlice(f.HelmValueFiles),
HelmFileValues: getStringSlice(f.HelmFileValues),
HelmStringValues: getStringSlice(f.HelmStringValues),
TerraformTFVars: getStringSlice(f.TerraformTFVars),
CloudFormationParamVars: getStringSlice(f.CloudformationParamVars),
TfExcludeDownloaded: getBool(f.TerraformExcludeDownloaded),
}, nil
}
Loading