Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Parallelize the chunked dispatches in reachable resources
This changes the datastore query to no longer have a limit, and instead produce a series of chunks for parallelized processing using the same parallelization code as the entrypoint cursor parallelization system. This is done in service of performance: a single database query is faster than making multiple roundtrips. This change also removes the limits on dispatching reachable resources and lookup resources in favor of grabbing as many results as possible and canceling dispatch once the overall limit has been reached. While this results in slightly more work being done than is necessary, it is far more performant from the caller's perspective (especially for no-limit calls), because there is no need to re-enter the reachable resources dispatch tree. As per the above change around limits, this change also further adjusts how context cancelation functions and adds a slew of tests to cover these new cases
- Loading branch information