Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide --baseline driver command argument that accepts SARIF log baseline file. #2370

Closed
michaelcfanning opened this issue Jun 30, 2021 · 3 comments · Fixed by #2371 or #2372
Closed

Comments

@michaelcfanning
Copy link
Member

The rough behavior should look like this:

  • Analysis runs and produces a SARIF log file.
  • When present, the baseline SARIF log file specified in the --baseline is loaded.
  • The baseline log and newly produced log go through the result matching API.
  • The result matcher marks all results as updated, new, unchanged, or absent
  • If the --inline switch is passed, the baseline log location is overwritten, otherwise a new log is generated.
  • By default, any issues that are marked as absent in the baseline should be flushed before result matching. And so the output for any newly generated file will only include absent results that reflect issues that have been between the baseline run (where they existed) and the new run (where they do not).

As part of this change, we should be sure to update our input argument validation. --baseline + --inline is sufficient for processing, for example, but --baseline without --inline (and with no output log file argument specified) is not legal. Etc.

Note that after we complete this first work, there may be some more useful work we could do. For example, users could filter the newly generated file to some subset or arbitrary set of baseline states, e.g., 'persist newly introduced issues only'. We could consider changes to result codes or introduce other breaking mechanisms ('break on any newly introduced issue').

The change described here is foundational and stands as useful on its own, however.

@michaelcfanning
Copy link
Member Author

@yongyan-gh
Copy link
Collaborator

Will take a look at this issue.

@eddynaka
Copy link
Collaborator

With the PR's above, we added the basic baseline functionality.
Let's close this issue and open a new one with enhancements :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants