Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add --min-confidence #196

Merged
merged 3 commits into from
Nov 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 28 additions & 13 deletions docs/snippets/help.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,31 @@ Arguments:
<INPUTS>... The workflow filenames or directories to audit

Options:
-p, --pedantic Emit findings even when the context suggests an explicit security decision made by the user
-o, --offline Only perform audits that don't require network access
-v, --verbose... Increase logging verbosity
-q, --quiet... Decrease logging verbosity
-n, --no-progress Disable the progress bar. This is useful primarily when running with a high verbosity level, as the two will fight for stderr
--gh-token <GH_TOKEN> The GitHub API token to use [env: GH_TOKEN=]
--format <FORMAT> The output format to emit. By default, plain text will be emitted [possible values: plain, json, sarif]
-c, --config <CONFIG> The configuration file to load. By default, any config will be discovered relative to $CWD
--no-config Disable all configuration loading
--no-exit-codes Disable all error codes besides success and tool failure
--min-severity <MIN_SEVERITY> Filter all results below this severity [possible values: unknown, informational, low, medium, high]
-h, --help Print help
-V, --version Print version
-p, --pedantic
Emit findings even when the context suggests an explicit security decision made by the user
-o, --offline
Only perform audits that don't require network access
-v, --verbose...
Increase logging verbosity
-q, --quiet...
Decrease logging verbosity
-n, --no-progress
Disable the progress bar. This is useful primarily when running with a high verbosity level, as the two will fight for stderr
--gh-token <GH_TOKEN>
The GitHub API token to use [env: GH_TOKEN=]
--format <FORMAT>
The output format to emit. By default, plain text will be emitted [possible values: plain, json, sarif]
-c, --config <CONFIG>
The configuration file to load. By default, any config will be discovered relative to $CWD
--no-config
Disable all configuration loading
--no-exit-codes
Disable all error codes besides success and tool failure
--min-severity <MIN_SEVERITY>
Filter all results below this severity [possible values: unknown, informational, low, medium, high]
--min-confidence <MIN_CONFIDENCE>
Filter all results below this confidence [possible values: unknown, low, medium, high]
-h, --help
Print help
-V, --version
Print version
11 changes: 6 additions & 5 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,16 +73,17 @@ All other exit codes are currently reserved.

There are two straightforward ways to filter `zizmor`'s results:

1. If all you need is severity filtering (e.g. "I want only medium-severity
and above results"), then you can use the `--min-severity` flag:
1. If all you need is severity or confidence filtering (e.g. "I want only
medium-severity and/or medium-confidence and above results"), then you can use
the `--min-severity` and `--min-confidence` flags:

!!! tip

`--min-severity` is available in `v0.6.0` and later.
`--min-severity` and `--min-confidence` are available in `v0.6.0` and later.

```bash
# filter unknown, informational, and low findings
zizmor --min-severity=medium ...
# filter unknown, informational, and low findings with unknown, low confidence
zizmor --min-severity=medium --min-confidence=medium ...
```

2. If you need more advanced filtering (with nontrivial conditions or
Expand Down
4 changes: 3 additions & 1 deletion src/finding/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@ pub(crate) mod locate;

// TODO: Traits + more flexible models here.

#[derive(Copy, Clone, Debug, Default, Eq, Hash, PartialEq, Serialize, ValueEnum)]
#[derive(
Copy, Clone, Debug, Default, Eq, Hash, Ord, PartialOrd, PartialEq, Serialize, ValueEnum,
)]
pub(crate) enum Confidence {
#[default]
Unknown,
Expand Down
6 changes: 5 additions & 1 deletion src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use anyhow::{anyhow, Context, Result};
use audit::WorkflowAudit;
use clap::{Parser, ValueEnum};
use config::Config;
use finding::Severity;
use finding::{Confidence, Severity};
use indicatif::{ProgressBar, ProgressDrawTarget, ProgressStyle};
use owo_colors::OwoColorize;
use registry::{AuditRegistry, FindingRegistry, WorkflowRegistry};
Expand Down Expand Up @@ -68,6 +68,10 @@ struct App {
#[arg(long)]
min_severity: Option<Severity>,

/// Filter all results below this confidence.
#[arg(long)]
min_confidence: Option<Confidence>,

/// The workflow filenames or directories to audit.
#[arg(required = true)]
inputs: Vec<PathBuf>,
Expand Down
7 changes: 6 additions & 1 deletion src/registry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use std::{collections::HashMap, path::Path, process::ExitCode};
use crate::{
audit::WorkflowAudit,
config::Config,
finding::{Finding, Severity},
finding::{Confidence, Finding, Severity},
models::Workflow,
App,
};
Expand Down Expand Up @@ -113,6 +113,7 @@ impl AuditRegistry {
pub(crate) struct FindingRegistry<'a> {
config: &'a Config,
minimum_severity: Option<Severity>,
minimum_confidence: Option<Confidence>,
ignored: Vec<Finding<'a>>,
findings: Vec<Finding<'a>>,
highest_seen_severity: Option<Severity>,
Expand All @@ -123,6 +124,7 @@ impl<'a> FindingRegistry<'a> {
Self {
config,
minimum_severity: app.min_severity,
minimum_confidence: app.min_confidence,
ignored: Default::default(),
findings: Default::default(),
highest_seen_severity: None,
Expand All @@ -139,6 +141,9 @@ impl<'a> FindingRegistry<'a> {
|| self
.minimum_severity
.map_or(false, |min| min > finding.determinations.severity)
|| self
.minimum_confidence
.map_or(false, |min| min > finding.determinations.confidence)
|| self.config.ignores(&finding)
{
self.ignored.push(finding);
Expand Down