Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

scan queue API redesign #504

Closed
JonoYang opened this issue Jul 19, 2024 · 1 comment
Closed

scan queue API redesign #504

JonoYang opened this issue Jul 19, 2024 · 1 comment
Assignees
@JonoYang

Comments

@JonoYang
Copy link
Member

Currently, the scan queue API has two endpoints:

  • /api/scan_queue/get_next_download_url/
  • /api/scan_queue/update_status/

The worker gets the next download url from /api/scan_queue/get_next_download_url/ and then performs a scan on it, then uses the /api/scan_queue/update_status/ to send back results or to indicate a failure.

It may be more logical to organize the scan queue API such that the update_status functions are actions on the scannable_uri instances themselves.
@JonoYang JonoYang self-assigned this Jul 19, 2024
JonoYang added a commit that referenced this issue Jul 20, 2024
@JonoYang
Update scan_queue API #504
00bc672 
    * Handle webhook response from scancode.io

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 22, 2024
@JonoYang
Update scan_queue API #504
4dc11f4 
    * Handle webhook response from scancode.io

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 24, 2024
@JonoYang
Update scan_queue API #504
fa2163c 
    * Handle webhook response from scancode.io

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 24, 2024
@JonoYang
Update index_package_scan #504
c40ad04 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 24, 2024
@JonoYang
Remove unused imports #504
072672e 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 24, 2024
@JonoYang
Set lookup_field on ScannableURIViewSet #504
c674d63 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 26, 2024
@JonoYang
Add code to create a webhook url #504
e09e8c7 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 29, 2024
@JonoYang
Add PurldbUser model and manager #504
a5f2cac 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 30, 2024
@JonoYang
Add migration for PurldbUser #504
97b9790 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 30, 2024
@JonoYang
Add endpoint for package scan notification #504
8ee69b7 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 31, 2024
@JonoYang
Remove PurldbUser model #504
edb0ef1 
    * We are going to use user.id for now to avoid migration issues

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 31, 2024
@JonoYang
Fix imports #504
642b755 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 31, 2024
@JonoYang
Update file access settings #504
b159b23 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 31, 2024
@JonoYang
Use project_extra_data as is #504
bc7411b 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 31, 2024
@JonoYang
Add api_view decorator #504
e471385 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 31, 2024
@JonoYang
Remove duplicated code #504
3001d96 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 31, 2024
@JonoYang
Remove duplicated code #504
c23b526 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 31, 2024
@JonoYang
Create tests for API changes #504
ad3a4b4 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 31, 2024
@JonoYang
Update CHANGELOG.rst #504
5dc8124 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jul 31, 2024
@JonoYang
Update CHANGELOG.rst #504
499e357 
    * Add docstring for index_package_scan

Signed-off-by: Jono Yang <jyang@nexb.com>
@JonoYang
Copy link
Member Author

JonoYang commented Aug 1, 2024

We now have the API endpoint /api/scan_queue/index_package_scan/ that is used to accept the purldb scan worker scan project info when a package scan has completed. Whenever get_next_download_url() is called, a webhook url is created, which is used as the webhook subscription target for the purldb scan worker project.

@JonoYang JonoYang closed this as completed Aug 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JonoYang JonoYang

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@JonoYang