Skip to content

Commit

Permalink
feat(repo): support local repositories (#4890)
Browse files Browse the repository at this point in the history
* feat(repo): support local repositories

* fix tests

* test: fix client/server tests

* docs: update

* test: add fs tests

* test: do not update golden files if overridden

* docs: remove a comment about fs deprecation
  • Loading branch information
knqyf263 authored Jul 31, 2023
1 parent 3c19761 commit d19c7d9
Show file tree
Hide file tree
Showing 124 changed files with 454 additions and 446 deletions.
236 changes: 0 additions & 236 deletions docs/docs/target/git-repository.md

This file was deleted.

155 changes: 155 additions & 0 deletions docs/docs/target/repository.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# Code Repository

Scan your local or remote code repositories for

- Vulnerabilities
- Misconfigurations
- Secrets
- Licenses

By default, vulnerability and secret scanning are enabled, and you can configure that with `--scanners`.

```bash
$ trivy repo (REPO_PATH | REPO_URL)
```

For example, you can scan a local repository as below.

```bash
$ trivy repo ./
```

It's also possible to scan a single file.

```
$ trivy repo ./trivy-ci-test/Pipfile.lock
```

To scan remote code repositories, you need to specify the URL.

```bash
$ trivy repo https://github.com/aquasecurity/trivy-ci-test
```

## Rationale
`trivy repo` is designed to scan code repositories, and it is intended to be used for scanning local/remote repositories in your machine or in your CI environment.
Therefore, unlike container/VM image scanning, it targets lock files such as package-lock.json and does not target artifacts like JAR files, binary files, etc.
See [here](../scanner/vulnerability/language/index.md) for the detail.

## Scanners
### Vulnerabilities
It is enabled by default.
Trivy will look for vulnerabilities based on lock files such as Gemfile.lock and package-lock.json.
See [here](../scanner/vulnerability/index.md) for the detail.

```
$ trivy repo ~/src/github.com/aquasecurity/trivy-ci-test
```

<details>
<summary>Result</summary>

```
2020-06-01T17:06:58.652+0300 WARN OS is not detected and vulnerabilities in OS packages are not detected.
2020-06-01T17:06:58.652+0300 INFO Detecting pipenv vulnerabilities...
2020-06-01T17:06:58.691+0300 INFO Detecting cargo vulnerabilities...
Pipfile.lock
============
Total: 10 (UNKNOWN: 2, LOW: 0, MEDIUM: 6, HIGH: 2, CRITICAL: 0)
+---------------------+------------------+----------+-------------------+------------------------+------------------------------------+
| LIBRARY | VULNERABILITY ID | SEVERITY | INSTALLED VERSION | FIXED VERSION | TITLE |
+---------------------+------------------+----------+-------------------+------------------------+------------------------------------+
| django | CVE-2020-7471 | HIGH | 2.0.9 | 3.0.3, 2.2.10, 1.11.28 | django: potential |
| | | | | | SQL injection via |
| | | | | | StringAgg(delimiter) |
+ +------------------+----------+ +------------------------+------------------------------------+
| | CVE-2019-19844 | MEDIUM | | 3.0.1, 2.2.9, 1.11.27 | Django: crafted email address |
| | | | | | allows account takeover |
+ +------------------+ + +------------------------+------------------------------------+
| | CVE-2019-3498 | | | 2.1.5, 2.0.10, 1.11.18 | python-django: Content |
| | | | | | spoofing via URL path in |
| | | | | | default 404 page |
+ +------------------+ + +------------------------+------------------------------------+
| | CVE-2019-6975 | | | 2.1.6, 2.0.11, 1.11.19 | python-django: |
| | | | | | memory exhaustion in |
| | | | | | django.utils.numberformat.format() |
+---------------------+------------------+----------+-------------------+------------------------+------------------------------------+
...
```

</details>

### Misconfigurations
It is disabled by default and can be enabled with `--scanners config`.
See [here](../scanner/misconfiguration/index.md) for the detail.

```shell
$ trivy repo --scanners config [YOUR_REPO_URL]
```

### Secrets
It is enabled by default.
See [here](../scanner/secret.md) for the detail.

```shell
$ trivy repo [YOUR_REPO_URL]
```

### Licenses
It is disabled by default.
See [here](../scanner/license.md) for the detail.

```shell
$ trivy repo --scanners license [YOUR_REPO_URL]
```

## SBOM generation
Trivy can generate SBOM for code repositories.
See [here](../supply-chain/sbom.md) for the detail.

## References
The following flags and environmental variables are available for remote git repositories.

### Scanning a Branch

Pass a `--branch` argument with a valid branch name on the remote repository provided:

```
$ trivy repo --branch <branch-name> <repo-name>
```

### Scanning upto a Commit

Pass a `--commit` argument with a valid commit hash on the remote repository provided:

```
$ trivy repo --commit <commit-hash> <repo-name>
```

### Scanning a Tag

Pass a `--tag` argument with a valid tag on the remote repository provided:

```
$ trivy repo --tag <tag-name> <repo-name>
```

### Scanning Private Repositories
In order to scan private GitHub or GitLab repositories, the environment variable `GITHUB_TOKEN` or `GITLAB_TOKEN` must be set, respectively, with a valid token that has access to the private repository being scanned.

The `GITHUB_TOKEN` environment variable will take precedence over `GITLAB_TOKEN`, so if a private GitLab repository will be scanned, then `GITHUB_TOKEN` must be unset.

You can find how to generate your GitHub Token in the following [GitHub documentation.](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)

For example:

```
$ export GITHUB_TOKEN="your_private_github_token"
$ trivy repo <your private GitHub repo URL>
# or
$ export GITLAB_TOKEN="your_private_gitlab_token"
$ trivy repo <your private GitLab repo URL>
```
2 changes: 1 addition & 1 deletion docs/getting-started/coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,4 +55,4 @@ Trivy reads IaC & configuration languages for the purpose of misconfiguration de
- Azure ARM Template
- Helm Chart

For more information about checks [see here](../docs/misconfiguration/policy/builtin.md).
For more information about checks [see here](../docs/scanner/misconfiguration/policy/builtin.md).
2 changes: 2 additions & 0 deletions docs/getting-started/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,5 @@ See [here](../docs/configuration/reporting.md#converting).
### How to run Trivy under air-gapped environment?
See [here](../docs/advanced/air-gap.md).

### Why `trivy fs` and `trivy repo` does not scan JAR files for vulnerabilities?
See [here](../docs/target/repository.md#rationale).
24 changes: 12 additions & 12 deletions integration/client_server_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -236,21 +236,21 @@ func TestClientServer(t *testing.T) {
golden: "testdata/busybox-with-lockfile.json.golden",
},
{
name: "scan pox.xml with fs command in client/server mode",
name: "scan pox.xml with repo command in client/server mode",
args: csArgs{
Command: "fs",
Command: "repo",
RemoteAddrOption: "--server",
Target: "testdata/fixtures/fs/pom/",
Target: "testdata/fixtures/repo/pom/",
},
golden: "testdata/pom.json.golden",
},
{
name: "scan sample.pem with fs command in client/server mode",
name: "scan sample.pem with repo command in client/server mode",
args: csArgs{
Command: "fs",
Command: "repo",
RemoteAddrOption: "--server",
secretConfig: "testdata/fixtures/fs/secrets/trivy-secret.yaml",
Target: "testdata/fixtures/fs/secrets/",
secretConfig: "testdata/fixtures/repo/secrets/trivy-secret.yaml",
Target: "testdata/fixtures/repo/secrets/",
},
golden: "testdata/secrets.json.golden",
},
Expand Down Expand Up @@ -279,7 +279,7 @@ func TestClientServer(t *testing.T) {
err := execute(osArgs)
require.NoError(t, err)

compareReports(t, c.golden, outputFile)
compareReports(t, c.golden, outputFile, nil)
})
}
}
Expand Down Expand Up @@ -328,11 +328,11 @@ func TestClientServerWithFormat(t *testing.T) {
{
name: "scan secrets with ASFF template",
args: csArgs{
Command: "fs",
Command: "repo",
RemoteAddrOption: "--server",
Format: "template",
TemplatePath: "@../contrib/asff.tpl",
Target: "testdata/fixtures/fs/secrets/",
Target: "testdata/fixtures/repo/secrets/",
},
golden: "testdata/secrets.asff.golden",
},
Expand Down Expand Up @@ -501,7 +501,7 @@ func TestClientServerWithToken(t *testing.T) {
}

require.NoError(t, err, c.name)
compareReports(t, c.golden, outputFile)
compareReports(t, c.golden, outputFile, nil)
})
}
}
Expand All @@ -528,7 +528,7 @@ func TestClientServerWithRedis(t *testing.T) {
err := execute(osArgs)
require.NoError(t, err)

compareReports(t, golden, outputFile)
compareReports(t, golden, outputFile, nil)
})

// Terminate the Redis container
Expand Down
2 changes: 1 addition & 1 deletion integration/docker_engine_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,7 @@ func TestDockerEngine(t *testing.T) {
assert.NoError(t, err, tt.name)

// check for vulnerability output info
compareReports(t, tt.golden, output)
compareReports(t, tt.golden, output, nil)

// cleanup
_, err = cli.ImageRemove(ctx, tt.input, api.ImageRemoveOptions{
Expand Down
5 changes: 4 additions & 1 deletion integration/integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -199,9 +199,12 @@ func execute(osArgs []string) error {
return app.Execute()
}

func compareReports(t *testing.T, wantFile, gotFile string) {
func compareReports(t *testing.T, wantFile, gotFile string, override func(*types.Report)) {
want := readReport(t, wantFile)
got := readReport(t, gotFile)
if override != nil {
override(&want)
}
assert.Equal(t, want, got)
}

Expand Down
3 changes: 2 additions & 1 deletion integration/module_test.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
//go:build module_integration

package integration

import (
Expand Down Expand Up @@ -70,7 +71,7 @@ func TestModule(t *testing.T) {
}()

// Compare want and got
compareReports(t, tt.golden, outputFile)
compareReports(t, tt.golden, outputFile, nil)
})
}
}
Loading

0 comments on commit d19c7d9

Please sign in to comment.