Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add offline command line option (#1452) #1480

Merged
merged 27 commits into from
Dec 29, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
771960a
Merge pull request #1 from intel/master
anthonyharrison May 27, 2020
20d6262
Merge remote-tracking branch 'upstream/master'
anthonyharrison Jun 1, 2020
5ab2c87
Merge remote-tracking branch 'upstream/master'
anthonyharrison Jul 18, 2020
ea57d05
Merge remote-tracking branch 'upstream/master'
anthonyharrison Aug 20, 2020
c2a8d1c
Merge remote-tracking branch 'upstream/master'
anthonyharrison Sep 1, 2020
7035ae9
Merge remote-tracking branch 'upstream/master'
anthonyharrison Sep 2, 2020
aaba9eb
Merge remote-tracking branch 'upstream/master'
anthonyharrison Sep 18, 2020
3b91b0e
Merge remote-tracking branch 'upstream/master'
anthonyharrison Oct 18, 2020
13fa1a8
Merge remote-tracking branch 'upstream/master'
anthonyharrison Nov 8, 2020
5db21eb
Merge remote-tracking branch 'upstream/master'
anthonyharrison Mar 14, 2021
9418560
Merge branch 'main'
anthonyharrison Mar 14, 2021
0f3f754
Merge branch 'main'
anthonyharrison Mar 24, 2021
9e04dab
Merge remote-tracking branch 'refs/remotes/upstream/main'
anthonyharrison May 30, 2021
da7e34a
Merge branch 'intel:main' into master
anthonyharrison Jun 20, 2021
8c9ea48
Merge branch 'intel:main' into master
anthonyharrison Jun 27, 2021
9581317
Merge branch 'intel:main' into master
anthonyharrison Jul 12, 2021
c59de25
Merge branch 'intel:main' into master
anthonyharrison Jul 25, 2021
bf5908a
Merge branch 'intel:main' into master
anthonyharrison Sep 3, 2021
2505e10
Merge branch 'intel:main' into master
anthonyharrison Sep 23, 2021
589b93d
Merge branch 'intel:main' into master
anthonyharrison Oct 13, 2021
42757cb
Merge branch 'intel:main' into master
anthonyharrison Oct 27, 2021
0f519a3
Merge branch 'intel:main' into master
anthonyharrison Nov 3, 2021
4016283
Merge remote-tracking branch 'upstream/main'
anthonyharrison Dec 5, 2021
cd7b07a
Merge remote-tracking branch 'upstream/main'
anthonyharrison Dec 9, 2021
8fabcbc
feat: Add offline command line option (#1452)
anthonyharrison Dec 26, 2021
ea7130f
feat: Add offline command option (#1452)
anthonyharrison Dec 26, 2021
f36945a
Merge branch 'main' into pr_1452
terriko Dec 29, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ Usage:
-V, --version show program's version number and exit
--disable-version-check
skips checking for a new version
--offline operate in offline mode

CVE Data Download:
-n {json,api}, --nvd {json,api}
Expand Down Expand Up @@ -118,6 +119,8 @@ in the terminal and provide it as an input by running `cve-bin-tool -L pkg-list`
You can use `--config` option to provide configuration file for the tool. You can still override options specified in config file with command line arguments. See our sample config files in the
[test/config](https://github.com/intel/cve-bin-tool/blob/main/test/config/)

Specifying the `--offline` option when running a scan ensures that cve-bin-tool doesn't attempt to download the latest database files or to check for a newer version of the tool.

The 0.3.1 release is intended to be the last release to officially support
python 2.7; please switch to python 3.6+ for future releases and to use the
development tree. You can check [our CI configuration](https://github.com/intel/cve-bin-tool/blob/main/.github/workflows/pythonapp.yml) to see what versions of python we're explicitly testing.
Expand Down
42 changes: 35 additions & 7 deletions cve_bin_tool/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
import platform
import sys
import textwrap
import time
from collections import ChainMap
from typing import Dict

Expand Down Expand Up @@ -219,6 +220,11 @@ def main(argv=None):
action="store_true",
help="skips checking for a new version",
)
parser.add_argument(
"--offline",
action="store_true",
help="operate in offline mode",
)

merge_report_group = parser.add_argument_group(
"Merge Report", "Arguments related to Intermediate and Merged Reports"
Expand Down Expand Up @@ -303,6 +309,7 @@ def main(argv=None):
"affected_versions": 0,
"sbom": "spdx",
"sbom_file": "",
"offline": False,
}

with ErrorHandler(mode=ErrorMode.NoTrace):
Expand Down Expand Up @@ -349,6 +356,15 @@ def main(argv=None):
if int(args["cvss"]) > 0:
score = int(args["cvss"])

# Offline processing
if args["offline"]:
# Override version check and database update arguments
version_check = True
db_update = "never"
else:
version_check = args["disable_version_check"]
db_update = args["update"]

# Check for PDF support
output_format = args["format"]
if output_format == "pdf" and importlib.util.find_spec("reportlab") is None:
Expand Down Expand Up @@ -380,12 +396,10 @@ def main(argv=None):
# Database update related settings
# Connect to the database
cvedb_orig = CVEDB(
version_check=not args["disable_version_check"],
version_check=not version_check,
error_mode=error_mode,
nvd_type=args["nvd"],
incremental_update=True
if args["update"] == "latest" and args["nvd"]
else False,
incremental_update=True if db_update == "latest" and args["nvd"] else False,
)

# if OLD_CACHE_DIR (from cvedb.py) exists, print warning
Expand All @@ -394,15 +408,23 @@ def main(argv=None):
f"Obsolete cache dir {OLD_CACHE_DIR} is no longer needed and can be removed."
)

# Check database exists if operating in offline mode.
if args["offline"] and not cvedb_orig.check_db_exists():
LOGGER.critical("Database does not exist.")
LOGGER.info(
"Consult the documentation at https://cve-bin-tool.readthedocs.io/en/latest/how_to_guides/offline.html to find out how to setup offline operation."
)
return -1

# Clear data if -u now is set
if args["update"] == "now":
if db_update == "now":
cvedb_orig.clear_cached_data()

if args["update"] == "latest":
if db_update == "latest":
cvedb_orig.refresh_cache_and_update_db()

# update db if needed
if args["update"] != "never":
if db_update != "never":
cvedb_orig.get_cvelist_if_stale()
else:
if args["nvd"] == "json":
Expand All @@ -417,6 +439,12 @@ def main(argv=None):
with ErrorHandler(mode=error_mode, logger=LOGGER):
raise CVEDataMissing("No data in CVE Database")

# Report time of last database update
db_date = time.strftime(
"%d %B %Y at %H:%M:%S", time.localtime(cvedb_orig.get_db_update_date())
)
LOGGER.info(f"CVE database last updated on {db_date}")

cvedb_orig.remove_cache_backup()

# Input validation
Expand Down
3 changes: 3 additions & 0 deletions cve_bin_tool/cvedb.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,9 @@ async def nist_fetch_using_api(self):
nvd_api.session = None
return nvd_api.all_cve_entries

def check_db_exists(self):
return os.path.isfile(self.dbpath)

def get_db_update_date(self):
# last time when CVE data was updated
self.time_of_last_update = datetime.datetime.fromtimestamp(
Expand Down
11 changes: 7 additions & 4 deletions doc/how_to_guides/offline.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,15 @@ NOTE The tool will error with InsufficientArgs because no directory was specifie
The way of transfer depends on the environment. The files to be transferred are in "~/.cache/cve-bin-tool"

## Import the vulnerability database file on the offline system
The vulnerability database should be copied into ~/.cache/cve-bin-tool
The vulnerability database should be copied into ~/.cache/cve-bin-tool.

## Run cve-bin-tool with --update never and --disable-version-check options
In an offline environment, when running a scan specify `--update never` so that cve-bin-tool doesn't attempt to download the latest database files and `--disable-version-check` so that the cve-bin-tool doesn't attempt to check for a newer version of the tool.
The cve-bin-tool will fail to operate in offline mode if a vulnerability database is not present on the system.

## Run cve-bin-tool with --offline option
In an offline environment, specify the `--offline` option when running a scan so that cve-bin-tool doesn't attempt to download the latest database files or check for a newer version of the tool.
The `--offline` option is equivalent to specifying `--update never` and `--disable-version-check` options.

## Maintenance Updates
In an offline environment, it is important to update the vulnerability database on a regular basis as often as you feel appropriate, so that the scanner can continue to detect recently-identified vulnerabilities. If any changes to CVE data is required (e.g. to remove false positives), you might also want to create and copy over a triage data file for usage.
In an offline environment, it is important to update the vulnerability database on a regular basis as often as you feel appropriate, so that the scanner can continue to detect recently-identified vulnerabilities. If any changes to CVE data is required (e.g. to remove false positives), you might also want to create and copy over a triage data file for usage. The time of the latest database update is reported whenever a scan is performed.

It is important to periodically check if the cve-bin-tool has also been updated as this check cannot be performed within an offline environment.