Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support environment.yaml files #6569

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/semantic-pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ jobs:
dart
swift
bitnami
conda

os
lang
Expand Down
36 changes: 36 additions & 0 deletions docs/docs/coverage/os/conda.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Conda

Trivy supports the following scanners for Conda packages.

| Scanner | Supported |
|:-------------:|:---------:|
| SBOM | ✓ |
| Vulnerability | - |
| License | ✓[^1] |


## SBOM
Trivy detects packages that have been installed with `Conda`.


### `<package>.json`
Trivy parses `<conda-root>/envs/<env>/conda-meta/<package>.json` files to find the version and license for the dependencies installed in your env.

### `environment.yml`[^2]
Trivy supports parsing [environment.yml][environment.yml][^2] files to find dependency list.

!!! note
License detection is currently not supported.

`environment.yml`[^2] files supports [version range][env-version-range]. We can't be sure about versions for these dependencies.
Therefore, you need to use `conda env export` command to get dependency list in `Conda` default format before scanning `environment.yml`[^2] file.

!!! note
For dependencies in a non-Conda format, Trivy doesn't include a version of them.


[^1]: License detection is only supported for `<package>.json` files
[^2]: Trivy supports both `yaml` and `yml` extensions.

[environment.yml]: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#sharing-an-environment
[env-version-range]: https://docs.conda.io/projects/conda-build/en/latest/resources/package-spec.html#examples-of-package-specs
35 changes: 18 additions & 17 deletions docs/docs/coverage/os/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,23 +9,24 @@ Trivy supports operating systems for

## Supported OS

| OS | Supported Versions | Package Managers |
|-----------------------------------------------|-------------------------------------|------------------|
| [Alpine Linux](alpine.md) | 2.2 - 2.7, 3.0 - 3.19, edge | apk |
| [Wolfi Linux](wolfi.md) | (n/a) | apk |
| [Chainguard](chainguard.md) | (n/a) | apk |
| [Red Hat Enterprise Linux](rhel.md) | 6, 7, 8 | dnf/yum/rpm |
| [CentOS](centos.md)[^1] | 6, 7, 8 | dnf/yum/rpm |
| [AlmaLinux](alma.md) | 8, 9 | dnf/yum/rpm |
| [Rocky Linux](rocky.md) | 8, 9 | dnf/yum/rpm |
| [Oracle Linux](oracle.md) | 5, 6, 7, 8 | dnf/yum/rpm |
| [CBL-Mariner](cbl-mariner.md) | 1.0, 2.0 | dnf/yum/rpm |
| [Amazon Linux](amazon.md) | 1, 2, 2023 | dnf/yum/rpm |
| [openSUSE Leap](suse.md) | 42, 15 | zypper/rpm |
| [SUSE Enterprise Linux](suse.md) | 11, 12, 15 | zypper/rpm |
| [Photon OS](photon.md) | 1.0, 2.0, 3.0, 4.0 | tndf/yum/rpm |
| [Debian GNU/Linux](debian.md) | 7, 8, 9, 10, 11, 12 | apt/dpkg |
| [Ubuntu](ubuntu.md) | All versions supported by Canonical | apt/dpkg |
| OS | Supported Versions | Package Managers |
|--------------------------------------|-------------------------------------|------------------|
| [Alpine Linux](alpine.md) | 2.2 - 2.7, 3.0 - 3.19, edge | apk |
| [Wolfi Linux](wolfi.md) | (n/a) | apk |
| [Chainguard](chainguard.md) | (n/a) | apk |
| [Red Hat Enterprise Linux](rhel.md) | 6, 7, 8 | dnf/yum/rpm |
| [CentOS](centos.md)[^1] | 6, 7, 8 | dnf/yum/rpm |
| [AlmaLinux](alma.md) | 8, 9 | dnf/yum/rpm |
| [Rocky Linux](rocky.md) | 8, 9 | dnf/yum/rpm |
| [Oracle Linux](oracle.md) | 5, 6, 7, 8 | dnf/yum/rpm |
| [CBL-Mariner](cbl-mariner.md) | 1.0, 2.0 | dnf/yum/rpm |
| [Amazon Linux](amazon.md) | 1, 2, 2023 | dnf/yum/rpm |
| [openSUSE Leap](suse.md) | 42, 15 | zypper/rpm |
| [SUSE Enterprise Linux](suse.md) | 11, 12, 15 | zypper/rpm |
| [Photon OS](photon.md) | 1.0, 2.0, 3.0, 4.0 | tndf/yum/rpm |
| [Debian GNU/Linux](debian.md) | 7, 8, 9, 10, 11, 12 | apt/dpkg |
| [Ubuntu](ubuntu.md) | All versions supported by Canonical | apt/dpkg |
| [OSs with installed Conda](conda.md) | - | conda |

## Supported container images

Expand Down
9 changes: 9 additions & 0 deletions integration/repo_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -341,6 +341,15 @@ func TestRepository(t *testing.T) {
},
golden: "testdata/conda-cyclonedx.json.golden",
},
{
name: "conda environment.yaml generating CycloneDX SBOM",
args: args{
command: "fs",
format: "cyclonedx",
input: "testdata/fixtures/repo/conda-environment",
},
golden: "testdata/conda-environment-cyclonedx.json.golden",
},
{
name: "pom.xml generating CycloneDX SBOM (with vulnerabilities)",
args: args{
Expand Down
80 changes: 80 additions & 0 deletions integration/testdata/conda-environment-cyclonedx.json.golden
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
{
"$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
"bomFormat": "CycloneDX",
"specVersion": "1.5",
"serialNumber": "urn:uuid:3ff14136-e09f-4df9-80ea-000000000004",
"version": 1,
"metadata": {
"timestamp": "2021-08-25T12:20:30+00:00",
"tools": {
"components": [
{
"type": "application",
"group": "aquasecurity",
"name": "trivy",
"version": "dev"
}
]
},
"component": {
"bom-ref": "3ff14136-e09f-4df9-80ea-000000000001",
"type": "application",
"name": "testdata/fixtures/repo/conda-environment",
"properties": [
{
"name": "aquasecurity:trivy:SchemaVersion",
"value": "2"
}
]
}
},
"components": [
{
"bom-ref": "3ff14136-e09f-4df9-80ea-000000000002",
"type": "application",
"name": "environment.yaml",
"properties": [
{
"name": "aquasecurity:trivy:Class",
"value": "lang-pkgs"
},
{
"name": "aquasecurity:trivy:Type",
"value": "conda-environment"
}
]
},
{
"bom-ref": "pkg:conda/bzip2@1.0.8",
"type": "library",
"name": "bzip2",
"version": "1.0.8",
"purl": "pkg:conda/bzip2@1.0.8",
"properties": [
{
"name": "aquasecurity:trivy:PkgType",
"value": "conda-environment"
}
]
}
],
"dependencies": [
{
"ref": "3ff14136-e09f-4df9-80ea-000000000001",
"dependsOn": [
"3ff14136-e09f-4df9-80ea-000000000002"
]
},
{
"ref": "3ff14136-e09f-4df9-80ea-000000000002",
"dependsOn": [
"pkg:conda/bzip2@1.0.8"
]
},
{
"ref": "pkg:conda/bzip2@1.0.8",
"dependsOn": []
}
],
"vulnerabilities": []
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
name: test-env
channels:
- defaults
dependencies:
- bzip2=1.0.8=h998d150_5
prefix: /opt/conda/envs/test-env
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ nav:
- CBL-Mariner: docs/coverage/os/cbl-mariner.md
- CentOS: docs/coverage/os/centos.md
- Chainguard: docs/coverage/os/chainguard.md
- Conda: docs/coverage/os/conda.md
- Debian: docs/coverage/os/debian.md
- Oracle Linux: docs/coverage/os/oracle.md
- Photon OS: docs/coverage/os/photon.md
Expand Down
103 changes: 103 additions & 0 deletions pkg/dependency/parser/conda/environment/parse.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
package environment

import (
"sort"
"strings"
"sync"

"golang.org/x/xerrors"
"gopkg.in/yaml.v3"

"github.com/aquasecurity/go-version/pkg/version"
"github.com/aquasecurity/trivy/pkg/dependency/types"
"github.com/aquasecurity/trivy/pkg/log"
xio "github.com/aquasecurity/trivy/pkg/x/io"
)

type environment struct {
Dependencies []Dependency `yaml:"dependencies"`
}

type Dependency struct {
Value string
Line int
}

type Parser struct {
logger *log.Logger
once sync.Once
}

func NewParser() types.Parser {
return &Parser{
logger: log.WithPrefix("conda"),
once: sync.Once{},
}
}

func (p *Parser) Parse(r xio.ReadSeekerAt) ([]types.Library, []types.Dependency, error) {
var env environment
if err := yaml.NewDecoder(r).Decode(&env); err != nil {
return nil, nil, xerrors.Errorf("unable to decode conda environment.yml file: %w", err)
}

var libs []types.Library
for _, dep := range env.Dependencies {
lib := p.toLibrary(dep)
// Skip empty libs
if lib.Name == "" {
continue
}
libs = append(libs, lib)
}

sort.Sort(types.Libraries(libs))
return libs, nil, nil
}

func (p *Parser) toLibrary(dep Dependency) types.Library {
name, ver := p.parseDependency(dep.Value)
if ver == "" {
p.once.Do(func() {
p.logger.Warn("Unable to detect the dependency versions from `environment.yml` as those versions are not pinned. Use `conda env export` to pin versions.")
})
}
return types.Library{
Name: name,
Version: ver,
Locations: types.Locations{
{
StartLine: dep.Line,
EndLine: dep.Line,
},
},
}
}

// parseDependency parses the dependency line and returns the name and the pinned version.
// The version range is not supported. It parses only the pinned version.
// e.g.
// - numpy 1.8.1
// - numpy ==1.8.1
// - numpy 1.8.1 py27_0
// - numpy=1.8.1=py27_0
//
// cf. https://docs.conda.io/projects/conda-build/en/latest/resources/package-spec.html#examples-of-package-specs
func (*Parser) parseDependency(line string) (string, string) {
line = strings.NewReplacer(">", " >", "<", " <", "=", " ").Replace(line)
parts := strings.Fields(line)
name := parts[0]
if len(parts) == 1 {
return name, ""
}
if _, err := version.Parse(parts[1]); err != nil {
return name, ""
}
return name, parts[1]
}

func (d *Dependency) UnmarshalYAML(node *yaml.Node) error {
d.Value = node.Value
d.Line = node.Line
return nil
}
Loading