Skip to content

Commit 1dedea5

Browse files
aegilopsaegilops
authored
Merge pull request #2 from advanced-security/copilot/refresh-documentation-scripts
Refresh documentation with current --help outputs and readable summaries
2 parents 87174bc + 05f4f8e commit 1dedea5

File tree

1 file changed

+108
-19
lines changed

1 file changed

+108
-19
lines changed

‎README.md‎

Lines changed: 108 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,14 @@ Run each specific script according to the help for each script.
2828

2929
### List secret scanning alerts
3030

31+
This script retrieves secret scanning alerts from GitHub repositories, organizations, or Enterprises and outputs them in CSV or JSON format. It supports filtering by state, date, and push protection bypass status. Use this to audit, analyze, or export secret scanning data for compliance or security purposes.
32+
3133
```text
32-
usage: list_secret_scanning_alerts.py [-h] [--scope {ent,org,repo}] [--bypassed] [--state {open,resolved}] [--no-include-secret] [--since SINCE] [--json] [--quote-all] [--hostname HOSTNAME] [--debug] name
34+
usage: list_secret_scanning_alerts.py [-h] [--scope {ent,org,repo}] [--generic] [--bypassed] [--state {open,resolved}]
35+
[--no-include-secret] [--include-locations] [--include-commit] [--since SINCE]
36+
[--json] [--raw] [--quote-all] [--hostname HOSTNAME]
37+
[--ca-cert-bundle CA_CERT_BUNDLE] [--no-verify-tls] [--quiet] [--debug]
38+
name
3339
3440
List secret scanning alerts for a GitHub repository, organization or Enterprise.
3541
@@ -40,23 +46,38 @@ options:
4046
-h, --help show this help message and exit
4147
--scope {ent,org,repo}
4248
Scope of the query
49+
--generic, -g Include generic secret types (not just vendor secret types/custom patterns, which is the
50+
default)
4351
--bypassed, -b Only show alerts where push protection was bypassed
4452
--state {open,resolved}, -s {open,resolved}
4553
State of the alerts to query
4654
--no-include-secret, -n
4755
Do not include the secret in the output
56+
--include-locations, -l
57+
Include locations in the output
58+
--include-commit, -c Include commit date and committer in the output
4859
--since SINCE, -S SINCE
49-
Only show alerts created after this date/time - ISO 8601 format, e.g. 2024-10-08 or 2024-10-08T12:00; or Nd format, e.g. 7d for 7 days ago
60+
Only show alerts created after this date/time - ISO 8601 format, e.g. 2024-10-08 or
61+
2024-10-08T12:00; or Nd format, e.g. 7d for 7 days ago
5062
--json Output in JSON format (otherwise CSV)
51-
--quote-all, -q Quote all fields in CSV output
63+
--raw, -r Output the raw data from the GitHub API
64+
--quote-all, -Q Quote all fields in CSV output
5265
--hostname HOSTNAME GitHub Enterprise hostname (defaults to github.com)
66+
--ca-cert-bundle CA_CERT_BUNDLE, -C CA_CERT_BUNDLE
67+
Path to CA certificate bundle in PEM format (e.g. for self-signed server certificates)
68+
--no-verify-tls Do not verify TLS connection certificates (warning: insecure)
69+
--quiet, -q Suppress non-error log messages
5370
--debug, -d Enable debug logging
5471
```
5572

5673
### List code scanning alerts
5774

75+
This script retrieves code scanning alerts from GitHub repositories, organizations, or Enterprises and outputs them in CSV or JSON format. It supports filtering by state and date. Use this to audit, track, or export code scanning findings for reporting and analysis.
76+
5877
```text
59-
usage: list_code_scanning_alerts.py [-h] [--scope {ent,org,repo}] [--state {open,resolved}] [--since SINCE] [--json] [--quote-all] [--hostname HOSTNAME] [--debug] name
78+
usage: list_code_scanning_alerts.py [-h] [--scope {ent,org,repo}] [--state {open,resolved}] [--since SINCE] [--json]
79+
[--raw] [--quote-all] [--hostname HOSTNAME] [--debug]
80+
name
6081
6182
List code scanning alerts for a GitHub repository, organization or Enterprise.
6283
@@ -70,19 +91,26 @@ options:
7091
--state {open,resolved}, -s {open,resolved}
7192
State of the alerts to query
7293
--since SINCE, -S SINCE
73-
Only show alerts created after this date/time - ISO 8601 format, e.g. 2024-10-08 or 2024-10-08T12:00; or Nd format, e.g. 7d for 7 days ago
94+
Only show alerts created after this date/time - ISO 8601 format, e.g. 2024-10-08 or
95+
2024-10-08T12:00; or Nd format, e.g. 7d for 7 days ago
7496
--json Output in JSON format (otherwise CSV)
97+
--raw, -r Output raw JSON data from the API
7598
--quote-all, -q Quote all fields in CSV output
7699
--hostname HOSTNAME GitHub Enterprise hostname (defaults to github.com)
77100
--debug, -d Enable debug logging
78101
```
79102

80103
### Replay code scanning alert status
81104

105+
This script replays or restores the status of code scanning alerts based on a previously exported CSV file. It's useful when alerts need to be re-dismissed after a repository is recreated or when migrating alert states between environments. The script reads from stdin and matches alerts by location.
106+
82107
```text
83-
usage: replay_code_scanning_alert_status.py [-h] [--scope {ent,org,repo}] [--state {open,resolved}] [--since SINCE] [--json] [--quote-all] [--hostname HOSTNAME] [--debug] name
108+
usage: replay_code_scanning_alert_status.py [-h] [--scope {ent,org,repo}] [--state {open,resolved}] [--since SINCE]
109+
[--json] [--quote-all] [--hostname HOSTNAME] [--debug]
110+
name
84111
85-
Replay code scanning alert status for a GitHub repository, organization or Enterprise, based on a provide file of previous statuses.
112+
Replay code scanning alert status for a GitHub repository, organization or Enterprise, based on a provide file of
113+
previous statuses.
86114
87115
positional arguments:
88116
name Name of the repo/org/Enterprise to query
@@ -94,7 +122,8 @@ options:
94122
--state {open,resolved}, -s {open,resolved}
95123
State of the alerts to query
96124
--since SINCE, -S SINCE
97-
Only show alerts created after this date/time - ISO 8601 format, e.g. 2024-10-08 or 2024-10-08T12:00; or Nd format, e.g. 7d for 7 days ago
125+
Only show alerts created after this date/time - ISO 8601 format, e.g. 2024-10-08 or
126+
2024-10-08T12:00; or Nd format, e.g. 7d for 7 days ago
98127
--json Output in JSON format (otherwise CSV)
99128
--quote-all, -q Quote all fields in CSV output
100129
--hostname HOSTNAME GitHub Enterprise hostname (defaults to github.com)
@@ -103,11 +132,17 @@ options:
103132

104133
### Replay secret scanning alert status
105134

135+
This script replays or restores the status of secret scanning alerts based on a previously exported CSV file. It's particularly useful when a repository is deleted and recreated, allowing you to restore the previous resolution states of alerts. The script reads a CSV file from stdin with columns: repo, secret, secret_type, state, resolution, resolution_comment, url.
136+
106137
```text
107-
usage: replay_secret_scanning_result_status.py [-h] [--scope {ent,org,repo}] [--state {open,resolved}] [--since SINCE] [--json] [--quote-all] [--hostname HOSTNAME] [--debug] name
138+
usage: replay_secret_scanning_result_status.py [-h] [--scope {ent,org,repo}] [--state {open,resolved}] [--since SINCE]
139+
[--json] [--quote-all] [--hostname HOSTNAME] [--debug]
140+
name
108141
109-
Replay secret scanning alert status for a GitHub repository, organization or Enterprise, based on a provided file of previous statuses. This can be useful if a repository is deleted and recreated, and you want to restore
110-
the previous status of the alerts. This script reads a CSV file with a header from stdin, with the following columns: repo, secret, secret_type, state, resolution, resolution_comment, url
142+
Replay secret scanning alert status for a GitHub repository, organization or Enterprise, based on a provided file of
143+
previous statuses. This can be useful if a repository is deleted and recreated, and you want to restore the previous
144+
status of the alerts. This script reads a CSV file with a header from stdin, with the following columns: repo, secret,
145+
secret_type, state, resolution, resolution_comment, url
111146
112147
positional arguments:
113148
name Name of the repo/org/Enterprise to query
@@ -119,7 +154,8 @@ options:
119154
--state {open,resolved}, -s {open,resolved}
120155
State of the alerts to query
121156
--since SINCE, -S SINCE
122-
Only show alerts created after this date/time - ISO 8601 format, e.g. 2024-10-08 or 2024-10-08T12:00; or Nd format, e.g. 7d for 7 days ago
157+
Only show alerts created after this date/time - ISO 8601 format, e.g. 2024-10-08 or
158+
2024-10-08T12:00; or Nd format, e.g. 7d for 7 days ago
123159
--json Output in JSON format (otherwise CSV)
124160
--quote-all, -q Quote all fields in CSV output
125161
--hostname HOSTNAME GitHub Enterprise hostname (defaults to github.com)
@@ -128,15 +164,21 @@ options:
128164

129165
### Enrich code scanning alerts
130166

167+
This script enriches code scanning alerts with additional CodeQL metadata, including query descriptions, MITRE CWE information, and other contextual details. It produces enhanced output in JSON, HTML, or PDF format, making it easier to generate comprehensive security reports. The alerts input must be in JSON format from `list_code_scanning_alerts.py`.
168+
131169
Using the PDF mode needs you to install `playwright`, which isn't in the general `requirements.txt`.
132170

133171
You can use `python3 -mpip install playwright` to get it, then run `playwright install` to install the required browsers.
134172

135173
```text
136-
usage: enrich_code_scanning_alerts.py [-h] [--mitre-cwe-csv MITRE_CWE_CSV] [--metadata-format {codeql,parse_ql}] [--debug] [--format {json,html,pdf}] [--fields FIELDS] [--groupby GROUPBY] alerts metadata scope
174+
usage: enrich_code_scanning_alerts.py [-h] [--mitre-cwe-csv MITRE_CWE_CSV] [--metadata-format {codeql,parse_ql}]
175+
[--debug] [--format {json,html,pdf}] [--fields FIELDS] [--groupby GROUPBY]
176+
alerts metadata scope
137177
138-
Add CodeQL metadata to Code Scanning alerts and produce output. This must be the abbreviated version of the JSON output supported by the partner script `list_code_scanning_alerts.py`. The metadata can either be in the format provided by the `codeql resolve metadata` command, or in the format produced by the
139-
script `parse_ql` by the same author as this script.
178+
Add CodeQL metadata to Code Scanning alerts and produce output. This must be the abbreviated version of the JSON
179+
output supported by the partner script `list_code_scanning_alerts.py`. The metadata can either be in the format
180+
provided by the `codeql resolve metadata` command, or in the format produced by the script `parse_ql` by the same
181+
author as this script.
140182
141183
positional arguments:
142184
alerts JSON file containing the alerts to enrich
@@ -146,11 +188,12 @@ positional arguments:
146188
options:
147189
-h, --help show this help message and exit
148190
--mitre-cwe-csv MITRE_CWE_CSV
149-
CSV file containing MITRE CWE data for Software Development from https://cwe.mitre.org/data/csv/699.csv.zip
191+
CSV file containing MITRE CWE data for Software Development from
192+
https://cwe.mitre.org/data/csv/699.csv.zip
150193
--metadata-format {codeql,parse_ql}, -m {codeql,parse_ql}
151194
Format of the metadata
152195
--debug, -d Print debug information
153-
--format {json,html}, -f {json,html}
196+
--format {json,html,pdf}, -f {json,html,pdf}
154197
Output format
155198
--fields FIELDS, -F FIELDS
156199
Comma-separated list of fields to include in the output
@@ -160,8 +203,13 @@ options:
160203

161204
### Resolve duplicate secret scanning alerts
162205

206+
This script identifies and resolves duplicate secret scanning alerts that occur when the same secret is detected by multiple patterns. For example, when a Google Cloud private key ID is detected both as a standalone secret and as part of service account credentials, this script can automatically resolve the duplicate. Use the `--add-matching-secret` option to add custom pairs of matching secret types.
207+
163208
```text
164-
usage: resolve_duplicate_secret_scanning_alerts.py [-h] [--scope {ent,org,repo}] [--state {open,resolved}] [--since SINCE] [--hostname HOSTNAME] [--debug] [--add-matching-secret OLD_TYPE NEW_TYPE] name
209+
usage: resolve_duplicate_secret_scanning_alerts.py [-h] [--scope {ent,org,repo}] [--state {open,resolved}]
210+
[--since SINCE] [--hostname HOSTNAME] [--debug]
211+
[--add-matching-secret OLD_TYPE NEW_TYPE]
212+
name
165213
166214
Resolve duplicate secret scanning alerts for a GitHub repository, organization or Enterprise.
167215
@@ -175,13 +223,54 @@ options:
175223
--state {open,resolved}, -s {open,resolved}
176224
State of the alerts to query
177225
--since SINCE, -S SINCE
178-
Only show alerts created after this date/time - ISO 8601 format, e.g. 2024-10-08 or 2024-10-08T12:00; or Nd format, e.g. 7d for 7 days ago
226+
Only show alerts created after this date/time - ISO 8601 format, e.g. 2024-10-08 or
227+
2024-10-08T12:00; or Nd format, e.g. 7d for 7 days ago
179228
--hostname HOSTNAME GitHub Enterprise hostname (defaults to github.com)
180229
--debug, -d Enable debug logging
181230
--add-matching-secret OLD_TYPE NEW_TYPE, -a OLD_TYPE NEW_TYPE
182231
Add a new pair of matched secret types
183232
```
184233

234+
### Close code scanning alerts
235+
236+
This script bulk-closes all open code scanning alerts for a specified repository. It's useful for cleanup operations, such as dismissing false positives or marking alerts as "won't fix" across an entire repository. The script supports dry-run mode to preview changes before applying them.
237+
238+
```text
239+
usage: close_code_scanning_alerts.py [-h] [--resolution {false positive,won't fix,used in tests}] [--dry-run] [-d]
240+
repo_name
241+
242+
Close all open code scanning alerts for a repository.
243+
244+
positional arguments:
245+
repo_name The owner/repo of the repository to close alerts for.
246+
247+
options:
248+
-h, --help show this help message and exit
249+
--resolution {false positive,won't fix,used in tests}
250+
The resolution of the alert.
251+
--dry-run Print the alerts that would be closed, but don't actually close them.
252+
-d, --debug Print debug messages to the console.
253+
```
254+
255+
### Estimate push protection rate
256+
257+
This script estimates what percentage of previously detected secrets would have been caught by push protection if it had been enabled. It compares a list of historical secret detections against the current patterns that have push protection enabled, helping you understand the potential impact of enabling this feature.
258+
259+
```text
260+
usage: estimate_push_protection_rate.py [-h] [--cut-off-date CUT_OFF_DATE] secrets_file patterns_file
261+
262+
Estimate push protection rate for secrets
263+
264+
positional arguments:
265+
secrets_file Path to the file containing the list of secrets
266+
patterns_file Path to the file containing the list of patterns with push protection
267+
268+
options:
269+
-h, --help show this help message and exit
270+
--cut-off-date CUT_OFF_DATE
271+
ISO date string to filter secrets detected after this date (e.g., 2023-01-01)
272+
```
273+
185274
## License
186275

187276
(C) Copyright 2024 GitHub, Inc. This is not open source software, and comes with no support or commitments.

0 commit comments

Comments
 (0)