Error when update

[strunz983]

Hello i get this error message when i try to update database my OS is
Linux kali 6.11.2-amd64 #1 SMP PREEMPT_DYNAMIC Kali 6.11.2-1kali1 (2024-10-15) x86_64 GNU/Linux

pip list 1 ⨯
Package Version

---

anyio 4.3.0
certifi 2024.2.2
exceptiongroup 1.2.0
fake-useragent 1.5.1
h11 0.14.0
httpcore 1.0.4
httpx 0.27.0
idna 3.6
pip 24.3.1
pyrate-limiter 2.10.0
sniffio 1.3.1
tqdm 4.66.2
typing_extensions 4.10.0

This is error

(venv)(root💀kali)-[~/CVEScannerV2/extra]
└─# python3 database.py -d cve.db

CVEScannerV2 Copyright (C) 2022-2024 Sergio Chica Manjarrez @ pervasive.it.uc3m.es.
Universidad Carlos III de Madrid.
This program comes with ABSOLUTELY NO WARRANTY; for details check below.
This is free software, and you are welcome to redistribute it
under certain conditions; check below for details.

[] Updating database...
[] Retrieving CVEs/CPEs metadata...
[+] Metadata: 2043 CPEs | 245652 CVEs
[+] Retrieving CPEs: 0%| | 0/2043 [00:00<?, ?it/s]Traceback (most recent call last):
File "/root/stuff/CVEScannerV2/extra/database.py", line 420, in query_api
data = resp.json()
^^^^^^^^^^^
File "/root/stuff/CVEScannerV2/extra/venv/lib/python3.12/site-packages/httpx/_models.py", line 764, in json
return jsonlib.loads(self.content, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/json/init.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/json/decoder.py", line 338, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/json/decoder.py", line 356, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Thank you

[scmanjarrez]

Hi, I think there is a problem with the API, I'm getting inconsistent results when querying it. I'll contact NVD and update you on the status.

[scmanjarrez]

This is the response from NVD regarding API problems:

As indicated on our communications page https://www.nist.gov/itl/nvd, we are experiencing a large volume of API requests. This is, in part, due to a large update to the entire dataset as part of supporting ADP data from the CVE List. We are aware of stability issues experienced by users of the APIs and are working to make improvements where we are able. 

The full communication about this is available on the provided website.

November 15, 2024:  NVD Technical Update
CVE List Authorized Data Publisher (ADP) Support
We plan to deploy changes to our systems the week of November 18th. After this is complete, NVD systems will begin ingesting supported datatypes within the CVE List from all sources (CNAs and ADPs). 

What does this mean?
CVE records within the NVD dataset will contain more information (Reference(s), CWE, and CVSS) from additional sources. This new information will be displayed on the website and in the API responses, attributed to the organization who contributed the information. More information regarding ADPs can be reviewed at https://www.cve.org/ProgramOrganization/ADPs

    .

    Downstream data consumers will notice a large shift in the volume of CVE Record modifications as part of this change. Going forward, organizations should expect CVE records to update at a higher frequency.

    Other relevant changes:
    Duplicate References and Reference Tags

    As part of NVD enrichment efforts, reference tags are associated with each reference provided by a specific source. In instances where the same reference is provided by multiple sources, any reference tags associated to an existing reference will be applied to the newly provided, duplicate reference automatically.

    Changes to NVD CVE Record Change History
        Event Names are now more consistently ordered when they are recorded at the same timestamp.
        Event Content (Actions and Change Types) will now be more consistently ordered.
        Reference and Reference Tag (Type) changes will now be audited separately across all cases.
        “CVE Received” Events will be re-labeled as “New CVE Received.” Using the “CVE Received” eventName parameter for the /cvehistory/ API will still return the appropriate results.

    CVE API and Vulnerability Search Impacts
    Due to upstream removal of data points used by the NVD systems, the following parameters will no longer filter search results. 
        CVE API: HasCertAlerts, HasCertNotes, HasOval
        Vulnerability Search:  US-CERT Technical Alerts, US-CERT Vulnerability Notes, OVAL Queries

    These options will be removed in a future release.

        Legacy Data Feed Files (1.1 JSON)
        While the json data provided by the 2.0 API will reflect the ADP updates immediately, the legacy data feed file updates will be staggered over a series of days.

[strunz983]

Thank you scmanjarrez for your help.
So we need to wait 😊

[scmanjarrez]

Hey, I’ve updated the database following the recent NVD update, as it seemed to be in a corrupted state. Could you take a look at the new database? You can either build your own using the repository CVEScannerV2DB, or grab a copy from the docker container:

docker run --rm -it -v /tmp/db:/tmp/db --entrypoint sh scmanjarrez/cvescanner
$ cp cve.db /tmp/db

Then you can get the copy from your /tmp/db directory

[strunz983]

Happy new year
Thank you scmanjarrez I'm gonna take a look 😊

[scmanjarrez]

Hey, is the problem fixed?

[strunz983]

Hello I just make update and i got this error message.
Thank you :)