-
Notifications
You must be signed in to change notification settings - Fork 173
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add batch querying for clearly defined to reduce ingestion time (#2088)
* change CD certifier to batch query Signed-off-by: pxp928 <parth.psu@gmail.com> * rebase ingestor scanner to use certifier Signed-off-by: pxp928 <parth.psu@gmail.com> * update unit tests and change ent to not index on attribution for clearlyDefined Signed-off-by: pxp928 <parth.psu@gmail.com> * add query rate limit per service for certifier Signed-off-by: pxp928 <parth.psu@gmail.com> * add query limit for license scanner on ingestion Signed-off-by: pxp928 <parth.psu@gmail.com> * fix and order unit test for certifiers Signed-off-by: pxp928 <parth.psu@gmail.com> * add comments to functions Signed-off-by: pxp928 <parth.psu@gmail.com> * fix unit tests Signed-off-by: pxp928 <parth.psu@gmail.com> * add re-try for clearly defined Signed-off-by: pxp928 <parth.psu@gmail.com> * add constant query size for CD and OSV Signed-off-by: pxp928 <parth.psu@gmail.com> --------- Signed-off-by: pxp928 <parth.psu@gmail.com>
- Loading branch information
Showing
28 changed files
with
947 additions
and
15,804 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.