Skip to content

Conversation

@michaelkedar
Copy link
Member

@michaelkedar michaelkedar commented Aug 7, 2025

Trying out the new GCS-backed vulnerabilities for making API queries on the staging instance (only). This should give us a good idea on the performance improvements.

  • Created server_new.py to (temporarily) keep the new matching logic a bit separate
  • Added new required indexes to allow AffectedVersions to be ordered by ID
  • Modified the page token to have some metadata that keeps track of the last returned ID (to avoid duplicate vulns across pages)

@michaelkedar michaelkedar marked this pull request as ready for review August 12, 2025 02:59
Copy link
Contributor

@another-rex another-rex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good to me


import osv

CURSOR_LAST_ID = 'last_id'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm can this be done at the metadata's type level? I.e. make metadata a struct with specific fields, and parse it into those fields, rather than having a dict that anyone can fill with anything. This way we can also limit the length of the ID to something reasonable as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've turned metadata into a dataclass, and made it raise the invalid page token error for unknown keys.
I didn't feel like doing the length check, since it isn't passed directly to datastore anyway.

@michaelkedar michaelkedar merged commit 0223374 into google:master Aug 13, 2025
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants