Release 2.3.0

🎉 We’re really excited to unveil Vulnerability-Lookup 2.3.0—our festive Christmas Release! 🎄

It includes new features, new importers, improvements and fixes.

✨ What's New

• Pub/Sub Mechanism:
  A streaming service implemented using a Publish/Subscribe (Pub/Sub) pattern powered by Valkey.
  Available channels: vulnerability, comment, bundle, and sighting.
  For more details, see the documentation (#92).

  • FediVuln now supports streaming information from a Valkey Pub/Sub service or an authenticated HTTP event-stream (both provided by Vulnerability-Lookup internals) to the Fediverse. Templates are used to render statuses, with the appropriate template selected based on the channel where the event originates.

• CISA Vulnrichment importer (as meta for CVE):
  The CISA Vulnrichment project is the public repository of CISA's enrichment
  of public CVE records through CISA's ADP (Authorized Data Publisher) container.
  In this phase of the project, CISA is assessing new and recent CVEs and adding
  key Stakeholder-Specific Vulnerability Categorization (SSVC)
  decision points. Once scored, some higher-risk CVEs will also receive
  enrichment of CWE and/or CVSS data points, where possible.
  The web interface will highlight this information related to the CVEs in the next release
  More information. (#42)

• CWE (Common Weakness Enumeration) and CAPEC (Common Attack Pattern Enumeration and Classification) importers (#97)

• New NCSC-NL CSAF Importer: (#94)

• New Route:
  /api/vulnerability/cpesearch/<string:cpe> to retrieve vulnerabilities by CPE (Common Platform Enumeration). (41f8471)

• New Website:
  A brand-new website featuring announcements and official documentation:
  https://www.vulnerability-lookup.org

🛠️ Changes

• Improved lookup for the cvelistv5_view macro description. (f4a929c)

• Added the ability for users to specify a source for sightings. (2be4eef)

• Updated kvrocks configuration with Docker support. (f864138)

• Added new with_linked, with_comments, with_bundles, and with_sightings arguments to the Vulnerability resource for the GET method. (8cb595)

Fixes

• Fail fast if Valkey/Redis fails to start. (#93)

• Fixed various minor issues in the HTML templates.

🙏 Thank you very much to all the contributors and testers!

Funding

The NGSOTI project is dedicated to training the next generation of Security Operation Center (SOC) operators, focusing on the human aspect of cybersecurity. It underscores the significance of providing SOC operators with the necessary skills and open-source tools to address challenges such as detection engineering, incident response, and threat intelligence analysis. Involving key partners such as CIRCL, Restena, Tenzir, and the University of Luxembourg, the project aims to establish a real operational infrastructure for practical training. This initiative integrates academic curricula with industry insights, offering hands-on experience in cyber ranges.

vulnerability-lookup is co-funded by CIRCL and by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or ECCC. Neither the European Union nor the granting authority can be held responsible for them.

Release 2.2.0

What's New

• Identity:

  • Vulnerability-Lookup now has a beautiful new logo.

• New Statistics Namespace:
  The API now offers a dedicated namespace for statistics. Two new endpoints are currently available:

  • /api/stats/vulnerability/most_sighted
  • /api/stats/vulnerability/most_commented
    Both endpoints provide the option to return results in a Markdown table format.
    (7a2b8ed,
    d95b49c)

  You can use the API output directly to generate PDF reports:

  $ curl -s -X 'GET' 'https://vulnerability.circl.lu/api/stats/vulnerability/most_sighted?date_from=2024-07-01&output=markdown' | pandoc --from=markdown --to=pdf -o semestrial-report.pdf

• New Client for KEV Sightings:
  A new Python client, KevSight, is available to generate sightings for Vulnerability-Lookup using the Known Exploited Vulnerabilities (KEV) catalog.
  For more details, refer to the Vulnerability-Lookup documentation on sightings automation and the available clients.

Changes

• API Enhancements:

  • Numerous improvements and harmonization across the API while maintaining
    compatibility with previous versions.
    PyVulVulnerabilityLookup has been updated. (f9a03fb, 79cc46d)

• Improved Views:

  • /recent: Enhanced for greater consistency and readability.
  • /vuln: Improved display for vulnerabilities from the CVE List v5 and NVD sources.
    The versions of the impacted products is now displayed in a third column. (9308772, 5f9826a, f71da45, 54ad96e, e8ae16e)
  • We have enhanced the layout and accessibility of various views as part of our ongoing
    commitment to creating software that is inclusive and usable for everyone.

• Documentation:

  • The documentation is now exposed by a dedicated Flask Blueprint which is serving static HTML
    files generated with Sphinx. Updating the documentation is possible via a simple button in the dashboard of Vulnerability-Lookup. (e27ac9f, 6f38ccf)

• Backend:

  • Harmonization of datetime objects by ensuring that all dates use UTC-aware timestamps (8ea2554,
    a4defc2)
  • Gunicorn is now by default using Gevent (c79f997)

Fixes

• Date Parameters in API:

  • The date_from and date_to parameters in the API no longer have default values.
    It is now the client’s responsibility to specify these values. (036ca3a)

• CVE Lookup Endpoint:

  • We fixed the API endpoint for searching CVEs by vendor and product. (0867fac)

Funding

The NGSOTI project is dedicated to training the next generation of Security Operation Center (SOC) operators, focusing on the human aspect of cybersecurity. It underscores the significance of providing SOC operators with the necessary skills and open-source tools to address challenges such as detection engineering, incident response, and threat intelligence analysis. Involving key partners such as CIRCL, Restena, Tenzir, and the University of Luxembourg, the project aims to establish a real operational infrastructure for practical training. This initiative integrates academic curricula with industry insights, offering hands-on experience in cyber ranges.

vulnerability-lookup is co-funded by CIRCL and by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or ECCC. Neither the European Union nor the granting authority can be held responsible for them.

Release 2.1.0

News

• A new API endpoint allows sightings to be created programmatically. Using dedicated tools, we gather observations from three main sources:
  • The Fediverse with FediVuln, a client to collect vulnerability-related information from the Fediverse
  • MISP with VulnerabilityLookupSighting, a client that retrieves vulnerability observations from a MISP server and pushes them to a Vulnerability Lookup instance
  • RSS and Atom feeds
• Combined sightings for bundles: The page displaying bundle details now shows the combined sightings for all vulnerabilities within the bundle (48610fc)
• New RSS/Atom endpoints for sightings have been added, allowing parameters such as the sighting source to be used and the id of a vulnerability. This enables users to subscribe to activity on unpublished vulnerabilities. (6020294)
• Provide the possibility to comment not yet published vulnerabilities (f88f239)

Changes

• Make drawBarChartHomePage faster (fa95945)
• Added a function in order to provide a small description about the vulnerabilities listed in the bundle page (8f04be1)
• Various enhancements to the home page and the charts based on user sightings
• Improved the API and the OpenAPI Swagger documentation
• Various improvements were made to the user interface

Fixes

• Fixed an issue in the Marshalling for the Sightings in the API (5ccdbe2)
• Fixed the title of the RSS/Atom feed (311d2c4)
• Keep non-sensitive case search even if our ids are lowercase. (9fae6ea)

Funding

The NGSOTI project is dedicated to training the next generation of Security Operation Center (SOC) operators, focusing on the human aspect of cybersecurity. It underscores the significance of providing SOC operators with the necessary skills and open-source tools to address challenges such as detection engineering, incident response, and threat intelligence analysis. Involving key partners such as CIRCL, Restena, Tenzir, and the University of Luxembourg, the project aims to establish a real operational infrastructure for practical training. This initiative integrates academic curricula with industry insights, offering hands-on experience in cyber ranges.

vulnerability-lookup is co-funded by CIRCL and by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or ECCC. Neither the European Union nor the granting authority can be held responsible for them.

Release 2.0.0

News

• Sightings: enable users to add observations to vulnerabilities with different types of sightings, such as: seen, exploited, not exploited, confirmed, not confirmed, patched, and not patched. (#76)

Changes

• Enhanced the homepage with simple charts displaying trending vulnerabilities based on user sightings. (ba7a64d, 946eaaf)
• The dump command can now export the sightings. (d070c58)
• Added a UUID attribute to the User model. A migration script will generate UUIDs for all existing users. (713a9a8)
• Improved /recent view for CSAF vulnerabilities. (f31b56)
• Improved /recent view for JVNDB vulnerabilities. (478faaf)
• Improved /recent view for OpenSSF Malicious Packages vulnerabilities. (1a2728d)
• Improved /recent view for PySec vulnerabilities. (f5d8b21)
• Improved /recent view for GitHub vulnerabilities. (e60b81a)
• Various graphical and accessibility improvements.

Fixes

• Get bigger chunks of variot data at a time (e9e6fce)
• [CISA feed] Exception when storing CWE entries as list (78236d5)

Funding

The NGSOTI project is dedicated to training the next generation of Security Operation Center (SOC) operators, focusing on the human aspect of cybersecurity. It underscores the significance of providing SOC operators with the necessary skills and open-source tools to address challenges such as detection engineering, incident response, and threat intelligence analysis. Involving key partners such as CIRCL, Restena, Tenzir, and the University of Luxembourg, the project aims to establish a real operational infrastructure for practical training. This initiative integrates academic curricula with industry insights, offering hands-on experience in cyber ranges.

vulnerability-lookup is co-funded by CIRCL and by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or ECCC. Neither the European Union nor the granting authority can be held responsible for them.

Release 1.7.0

News

• The sign-up process now verifies that the user is not using a disposable email address,
  thanks to the MISP warning list of disposable emails. (177f6a9)
• Added a Flask command to update the local MISP Warning Lists. This function is triggered
  automatically when the Vulnerability Lookup instance is updated. (5370aa5)
• Implemented a mechanism to execute maintenance background jobs directly from the admin panel. (f36053b)
• Integration of the Exploit Prediction Scoring System (EPSS) score. (9e54b71, 27b4487, 2e021ed).
• A theme switcher button with two modes: light mode (default) and dark mode. (09ab04a, 2e12ddf)

Changes

• Enhanced the formatting of comments and bundle descriptions using JavaScript. (ddedfca)
• Admins are now notified when a new comment is awaiting moderation. (9739292)
• All admin views were improved with better search capabilities.

Fixes

• Make the GET List (with filters) for Bundles in the API case insensitive. (32c9bb4)
• The search form must post the search to /search and not search. (4ce5227)

Funding

The NGSOTI project is dedicated to training the next generation of Security Operation Center (SOC) operators, focusing on the human aspect of cybersecurity. It underscores the significance of providing SOC operators with the necessary skills and open-source tools to address challenges such as detection engineering, incident response, and threat intelligence analysis. Involving key partners such as CIRCL, Restena, Tenzir, and the University of Luxembourg, the project aims to establish a real operational infrastructure for practical training. This initiative integrates academic curricula with industry insights, offering hands-on experience in cyber ranges.

vulnerability-lookup is co-funded by CIRCL and by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or ECCC. Neither the European Union nor the granting authority can be held responsible for them.

Release 1.6.0

News

• Importer for Tailscale vulnerabilities #68
• New user profile page with more information and detection of the country
  during user sign-up using the CIRCL MMDB service #73
• Added the ability to filter comments by any taxonomy tags by clicking on the corresponding badge (b3e0bdf)
• Implemented a function to back up the database using pg_dump. This function is automatically triggered by the
  project's update command, ensuring a backup is created before any database upgrades take place. (75ee913)

Improvements

• [API] Enhanced detection of CVE, GHSA, and PySec IDs within bundle descriptions and comments.
  This enables automatic identification of related vulnerabilities linked to a comment or a bundle. (2c00695, 162a599, 401d780).
• Added more validation to the various attributes of the User model. (758e571, 3a1cc60)
• Simplified search page (f2c55bc)
• Improved display of tables and lists generated from Markdown (in comments and bundles) (24fa4f9, 15fe9b2)
• The ranking of the users is now taking into account the contributions of comments and bundles.
  Users who have never contributed are sorted by last_seen, after the processed result. (4e4a436)
• Various graphical and accessibility improvements.

Fixes

• Do not iterate over meta tags when never defined in an object (93f9966)

Funding

The NGSOTI project is dedicated to training the next generation of Security Operation Center (SOC) operators, focusing on the human aspect of cybersecurity. It underscores the significance of providing SOC operators with the necessary skills and open-source tools to address challenges such as detection engineering, incident response, and threat intelligence analysis. Involving key partners such as CIRCL, Restena, Tenzir, and the University of Luxembourg, the project aims to establish a real operational infrastructure for practical training. This initiative integrates academic curricula with industry insights, offering hands-on experience in cyber ranges.

vulnerability-lookup is co-funded by CIRCL and by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or ECCC. Neither the European Union nor the granting authority can be held responsible for them.

Release 1.5.0

Release 1.5.0 (2024-08-30)

News

• new: Import JVN DB (Japan database of vulnerability countermeasure information).
  Closes #67
• new: [commands] The dump command can now dump comments (--comments) and bundles (--bundles).
  Closes #65.
• new: [website] Is now possible to assign tags related to a comment.
  Tags are automatically stored in the meta field of the comment. The default taxonomy used for the tags is defined in the MISP project: https://www.misp-project.org/taxonomies.html#_vulnerability_3

Improvements

• chg: [API] Added a new argument in order to let the user filter comments based on data in the meta JSON field.
• chg: [website] Filtering the list of public comments by their type (in the future with a taxonomy)
• chg: [website] Added references from the NVD meta section in the details view
• chg: [API] Added the possibility to filter bundles with a query on the meta JSONB field via the API.

Fixes

• fix: [website] Ensures that the vulnerability id has been specified by the user.
  even when the user is an administrator.

Online version

If you want to test vulnerability-lookup without installing it, you can use the CIRCL public instance https://vulnerability.circl.lu/.

Funding

The NGSOTI project is dedicated to training the next generation of Security Operation Center (SOC) operators, focusing on the human aspect of cybersecurity. It underscores the significance of providing SOC operators with the necessary skills and open-source tools to address challenges such as detection engineering, incident response, and threat intelligence analysis. Involving key partners such as CIRCL, Restena, Tenzir, and the University of Luxembourg, the project aims to establish a real operational infrastructure for practical training. This initiative integrates academic curricula with industry insights, offering hands-on experience in cyber ranges.

vulnerability-lookup is co-funded by CIRCL and by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or ECCC. Neither the European Union nor the granting authority can be held responsible for them.

Release 1.4.0

Release 1.4.0 (2024-08-02)

News

• new: [webservice] Display severity informations for vulnerabilities from GitHub, NVD and Pysec.
• new: [API] Added an endpoint to let the user regenerate the API key.
• new: [API] New endpoint to let an administrator delete a user and a new endpoint in order to return information about the currently authenticated user (GET /user/me)
• new: [API] Added new endpoint (/configInfo) which returns non-sensitive information about the configuration of the system.

Improvements

• chg: [website] Reorganization and improvements to diffenrent views that are dedicated to administrators. (27088c9)
• test suite improvements (a51a144, 88f6e25, 46775e7)
• a boolean in the config file can now specify whether two-factor authentication is enforced or not (fedf631)

Fixes

• Spaces in vulnerability id in the main search box doesn't work.
• empty recent

Funding

The NGSOTI project is dedicated to training the next generation of Security Operation Center (SOC) operators, focusing on the human aspect of cybersecurity. It underscores the significance of providing SOC operators with the necessary skills and open-source tools to address challenges such as detection engineering, incident response, and threat intelligence analysis. Involving key partners such as CIRCL, Restena, Tenzir, and the University of Luxembourg, the project aims to establish a real operational infrastructure for practical training. This initiative integrates academic curricula with industry insights, offering hands-on experience in cyber ranges.

vulnerability-lookup is co-funded by CIRCL and by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or ECCC. Neither the European Union nor the granting authority can be held responsible for them.

Release 1.3.0

Release 1.3.0 (26-07-2024)

Improvements

• Vulnerability Details Page Enhancements: We've significantly enhanced the vulnerabilities details page. It now presents more relevant information and the layout has been substantially improved for a better user experience.
• API Enhancements: Various improvements have been made to the API for better performance and functionality.
• UI Enhancements: Edition/action buttons are now hidden when not logged in (#57).
• Importer Improvements: Enhancements have been made to various importers (37d3a6d).

Fixes

• Custom Vulnerability Display Bug: Fixed an issue where custom vulnerabilities were not displayed correctly (#58).
• New Vulnerability Creation Issue: Resolved the problem where new vulnerabilities couldn't be created without a CVE number (#56).
• Webservice Sorting Fix: Fixed the sorting issue of contributors versus users (46195d1).
• Minor Fixes: Various minor fixes have been implemented to improve overall stability and performance.

Funding

The NGSOTI project is dedicated to training the next generation of Security Operation Center (SOC) operators, focusing on the human aspect of cybersecurity. It underscores the significance of providing SOC operators with the necessary skills and open-source tools to address challenges such as detection engineering, incident response, and threat intelligence analysis. Involving key partners such as CIRCL, Restena, Tenzir, and the University of Luxembourg, the project aims to establish a real operational infrastructure for practical training. This initiative integrates academic curricula with industry insights, offering hands-on experience in cyber ranges.

vulnerability-lookup is co-funded by CIRCL and by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or ECCC. Neither the European Union nor the granting authority can be held responsible for them.

vulnerability-lookup version 1.2.0 released with new bundle features, many API improvements, UI and many other improvements

vulnerability-lookup version 1.2.0 released with new bundle features, many API improvements, UI and many other improvements

Notable Changes

Bundles Management

• Introduced a new SQLAlchemy model, Bundle, for the website.
  • Included migration script and automatic execution of migrations via the app update script. Details here.
• Added a new API endpoint for managing bundles:
  • Only administrators can create, edit, and delete bundles.
  • Bundles are always public.
• Added a third tab in the vulnerability_view page (/vuln/vuln-id) to list bundles referencing the current vulnerability.
• Introduced a new view (/bundle/<bundle-uuid>) presenting bundle details with a share button (Reddit, Newspipe, Pinboard).

User Profile Page

• Implemented a new column-based layout with data loaded via the API.
• Displayed recent comments and bundles owned by a user in two columns with dedicated RSS/ATOM feeds.
• Added new RSS/ATOM feeds to publish user activity (/user/<login>.atom or /user/<login>.rss).

Minor Fixes

• Fixed: Name of vendors returned by get_vendors are now lowercase, addressing an issue in vendor search.
• Fixed: vulnerabilitylookup.get_vendors returns lowercase results.
• Various other minor fixes.

Minor Changes

• Enabled CSRF globally.
• Added a datalist refresh system in JavaScript for the freetext input field on the home page.
• Rendered Markdown code from GitHub, PySec, and VarIoT advisories.
• Backend no longer validates JSON vulnerability data sent from an admin.
• Implemented a blocklist for usernames to prevent undesirable words.
• Replaced moment.js with Luxon.
• Added the option to specify a vendor ID in the feed of recent vulnerabilities.
• Displayed the number of elements in each tab on the /vuln/<vuln-id> page.
• Improved the About page.

Additional Improvements

• Enhanced the display of the /recent pages with better column sizing for easier readability.
• Harmonized the format and data of various RSS/ATOM feeds.
• Made improvements to the users directory page.
• Made various improvements to the API, including harmonization of query parameters, result marshaling, and pagination.
• Other small changes and enhancements.

Funding

The NGSOTI project is dedicated to training the next generation of Security Operation Center (SOC) operators, focusing on the human aspect of cybersecurity. It underscores the significance of providing SOC operators with the necessary skills and open-source tools to address challenges such as detection engineering, incident response, and threat intelligence analysis. Involving key partners such as CIRCL, Restena, Tenzir, and the University of Luxembourg, the project aims to establish a real operational infrastructure for practical training. This initiative integrates academic curricula with industry insights, offering hands-on experience in cyber ranges.

vulnerability-lookup is co-funded by CIRCL and by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or ECCC. Neither the European Union nor the granting authority can be held responsible for them.