Speed up JudgeHandler's processing of problem list #328
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Type of change: improvement
What
Speed up JudgeHandler's processing of problem list
Why
OJ has more than 40k+ problems now. Judges sometimes stall waiting for
bridged
to process the problem list.When test data for any problem changes, judges send the codes of ALL judge-able problems to
bridged
, which then maps these codes to primary keys:This is extremely slow because:
set
runs.Mitigated by only querying the primary keys as values list.
A proper fix for this could be caching the mapping, but it's not straightforward as problems can be renamed.
How Has This Been Tested?
On my PC, I simulate processing all problems:
Previously, this takes 0.351s. After this patch, it only takes 0.129s :)
Be aware that my PC is faster than our server, and I only run 1 judge instead of 4. The processing time would be further multiplied by the number of judges.
Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the AGPL-3.0 License.