Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update imports.lock less frequently #296

Merged
merged 2 commits into from
Aug 22, 2022
Merged

Conversation

mystor
Copy link
Collaborator

@mystor mystor commented Aug 19, 2022

This patch reduces the frequency that imports.lock updates by doing a
few things:

  • Imports are only updated when necessary for the vet to pass. If vet
    would successfully pass without the new audits, only changes to
    notes, criteria descriptions, revoked audits, and violations will be
    pulled.

  • Some commands, such as suggest, will always run against the live
    copy when not --locked, providing the most accurate results possible.

  • Only audits for crates in your crate graph will be imported.

  • When an update is required, all new audits for the given crate will
    be fetched and locked, to batch updates as much as possible.

This required doing some changes to how imports are handled under the
hood, as well as a number of changes to the resolver in order to track
whether fresh imports are required with sufficient accuracy. Imports are
also now fetched in more APIs in order to have the most up-to-date
information.

The changes to the resolver involved piping through "caveats" which are
determined when solving paths. This is an expansion on the existing
"needs_exemption"/"fully_audited" flags which were being used to
generate stats after a successful vet to also handle freshly imported
audits which should only be used when necessary.

Special handling was also needed to make sure that audits for unused
criteria are not counted by tracking caveats on a per-criteria basis and
bubbling them out.

Fixes #272

These will be used in the imports tests to allow snapshotting diffs of the
relevant imports.lock files.
@mystor mystor requested a review from Gankra August 19, 2022 00:13
This patch reduces the frequency that imports.lock updates by doing a
few things:

 * Imports are only updated when necessary for the vet to pass. If vet
   would successfully pass without the new audits, only changes to
   notes, criteria descriptions, revoked audits, and violations will be
   pulled.

 * Some commands, such as suggest, will always run against the live
   copy when not --locked, providing the most accurate results possible.

 * Only audits for crates in your crate graph will be imported.

 * When an update is required, all new audits for the given crate will
   be fetched and locked, to batch updates as much as possible.

This required doing some changes to how imports are handled under the
hood, as well as a number of changes to the resolver in order to track
whether fresh imports are required with sufficient accuracy. Imports are
also now fetched in more APIs in order to have the most up-to-date
information.

The changes to the resolver involved piping through "caveats" which are
determined when solving paths. This is an expansion on the existing
"needs_exemption"/"fully_audited" flags which were being used to
generate stats after a successful vet to also handle freshly imported
audits which should only be used when necessary.

Special handling was also needed to make sure that audits for unused
criteria are not counted by tracking caveats on a per-criteria basis and
bubbling them out.
@bholley
Copy link
Collaborator

bholley commented Aug 22, 2022

Gankra gave this the thumbs-up.

@bholley bholley merged commit 2c96a2c into mozilla:main Aug 22, 2022
mystor added a commit to mystor/cargo-vet that referenced this pull request Aug 22, 2022
The changes in mozilla#296 changed some test expectations which were only added
in mozilla#299, so when both were landed, it caused test failures. This fixes
those test failures.
mystor added a commit that referenced this pull request Aug 22, 2022
The changes in #296 changed some test expectations which were only added
in #299, so when both were landed, it caused test failures. This fixes
those test failures.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Imported audits may change imports.lock each time cargo vet is run
2 participants