New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(crawl-status): consider concurrency limited jobs as prioritized (FIR-851) #1184

Merged

mogery merged 1 commit into main from mog/fix-early-complete

Feb 16, 2025

Member

mogery commented Feb 16, 2025

Fixes a multitude of annoying bugs, including:

Crawls that have all their pending jobs in the concurrency queue are considered "completed" (FIR-851)
Concurrency limited scrapes do not show up in the total immediately


          fix(crawl-status): consider concurrency limited jobs as prioritized

ec6236e

mogery requested a review from nickscamara

February 16, 2025 08:27

Member Author

mogery commented Feb 16, 2025

Did not write a test because it's really dumb hard to test for this. We should build a harness I think, eventually.

nickscamara approved these changes

View reviewed changes

mogery merged commit fd8b389 into main

1 check passed

mogery deleted the mog/fix-early-complete branch

February 16, 2025 14:52

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet