Parallel dependency source updates

[susitsm]

Problem

Currently, source updates are done in a blocking way during dependency resolution. With many different sources (many packages with git path in my case), this can become the bottleneck of cargo update.

Proposed Solution

While querying dependencies, ensure_loaded is called to make sure the dependency source is loaded. Changing it to return a Poll allows processing other dependencies, and their sources, while the source is being downloaded.

Progress bars

Since git sources use progress bars, it must be ensured that running these updates on multiple threads does not create a flickering progress bar (you can observe this in my example during the uncached benchmark, if ran without --quiet). Since the discussion on what and how cargo should display is still ongoing in #8889, I would implement a simple system with the goal of keeping close to the current output: It would show the last started progress bar until finished, then fall back to the previous. This also has the benefit of requiring no new terminal features.

Notes

This issue was moved from the internals forum. Addressing @epage's concerns:

If/when you bring this to the cargo team, it would be good to include reproduction steps as well as characterize the performance impact (when it happens, how slow it can be, and end-user impact).

These benchmarks were created using my example crate with 13 git sources

Starting "cargo update" benchmarks
Uncached, serial, cargo 1.91.0-nightly (840b83a10 2025-07-30):

real	0m18.497s
user	0m6.969s
sys	0m0.435s

Uncached, parallel:

real	0m6.372s
user	0m7.343s
sys	0m0.825s

Cached, serial, cargo 1.91.0-nightly (840b83a10 2025-07-30):

real	0m4.445s
user	0m0.707s
sys	0m0.101s

Cached, parallel:

real	0m1.036s
user	0m1.103s
sys	0m0.246s

With -Zgit=shallow-deps

Uncached, shallow, serial, cargo 1.91.0-nightly (840b83a10 2025-07-30):

real	0m9.295s
user	0m1.116s
sys	0m0.215s

Uncached, shallow, parallel:

real	0m1.521s
user	0m1.479s
sys	0m0.317s

The parallel updates are 2-6x faster than the serial ones. For the cached case, the git updates should be even faster, the bottleneck is updating crates.io dependencies.

In addition to the problems you gave, note that the function you linked to specifically calls out that it is blocking and why.

ensure_loaded is a private function, called in 3 other functions (query, patch and add_sources). Changing ensure_loaded to return a Poll (maybe even renaming it to pend_until_loaded) integrates nicely with query and patch, as they already handle Poll-s. add_sources will need a loop to wait for all pending sources. These changes will bring the performance boost of parallel source updates to all code using these 3 public functions, without a need for any changes.

We've been cautious about what terminal features we take advantage of for maximum compatibility. We've branched out more recently with unicode and hyperlink support. Like those, any progress improvements over what we have today will likely need some kind of end-user control and ideally auto-detection.

The current proposal uses no new terminal features and minimizes the changes to cargo's output

Not quite sure if GlobalContext needs to be thread safe for this. If something can fit within our pre-async async framework, then Sync shouldn't be needed, which describes most of our networking and Source logic.

Right, it is enough to convert the display part to Sync. Even though I think it would be benefical, as it would simplify parallelizing other things without noticable performance impact on serial code, that should be its own discussion.

[epage]

In addition to the problems you gave, note that the function you linked to specifically calls out that it is blocking and why.

You quoted this but I'm not seeing the motivation for it being blocking addressed.

[susitsm]

I should have expanded more on it:

I would say (or rather speculate, since I'm neither the author of the code or the comment) the main motivation for it blocking is that loading dependency summaries without finishing loading the source is not sound. This behavior should not change. Thankfully, ensure_loaded is a private function. The implementor only needs to handle Poll::Pending in the 3 callers. patch and add_sourcesmust not return while some sources are being loaded and query must immediately return a Poll::Pending if the source loading is still in progress.

Besides a correctness issue, there could be multiple other reasons:

1. Ease of implementation: I don't think there is any technical reason why add_sources or patch could not return a Poll, but exposing cargo's pre-async async framework is tedious to handle for the user.
2. Performance: The keen-eyed might have noticed that the full cpu time, user, has increased in my benchmarks. Since cargo's pre-async async framework lacks wakers, we must loop on one thread while the other finishes the download, which can take some time for larger repositories. The blocking version does not have this overhead.

[epage]

There is a comment where it blocks, saying why it blocks

        // This isn't strictly necessary since it will be called later.
        // However it improves error messages for sources that issue errors
        // in `block_until_ready` because the callers here have context about
        // which deps are being resolved.
        self.block_until_ready()?;

[epage]

include reproduction steps as well as characterize the performance impact (when it happens, how slow it can be, and end-user impact).

This was quoted from me but I'm not seeing it addressed. This should be the focus of the Problem while making things parallel is part of Proposed Solution. By focusing on workflows, we can better evaluate the impact of the Problem against the trade offs of a Solution. For instance, we cache the cloned repo so this isn't a problem on every run. So this is only a problem for new clones of your repo or when dependencies are updated. I assume this is primarily an issue for CI and caching of the cache isn't worth it because that is also network traffic.

Would you mind also looking at what the trade off is compared to shallow clones, see https://doc.rust-lang.org/1.89.0/cargo/reference/unstable.html#git

[susitsm]

Would you mind also looking at what the trade off is compared to shallow clones

Added benchmarks with Zgit=shallow-deps (there is a one-liner to reproduce my results in the linked example crate). Shallow clones and parallel updates complement each other well, with a staggering 1.5 sec uncached cargo update. No trade-offs.

Uncached, serial, cargo 1.91.0-nightly (840b83a10 2025-07-30):

real	0m18.497s
user	0m6.969s
sys	0m0.435s

Uncached, shallow, serial, cargo 1.91.0-nightly (840b83a10 2025-07-30):

real	0m9.295s
user	0m1.116s
sys	0m0.215s

Uncached, shallow, parallel:

real	0m1.521s
user	0m1.479s
sys	0m0.317s

[susitsm]

There is a comment where it blocks, saying why it blocks

I was looking at the doc comment of ensure_loaded. My mistake. I don't have an immediate answer to that, will have to look into it more.

I assume this is primarily an issue for CI and caching of the cache isn't worth it because that is also network traffic.

This is a development issue for me, not a CI one. Having git = "..", branch = ".." dependencies where the branches are frequently updated, running cargo update that fetches a few new commits can take 10-20 seconds depending on latency. Nevertheless, parallelizing source updates speeds up other workflows as well. It also seems to work especially well with Zgit=shallow-deps

User impact

I'm not sure what the expectations are about end user impact. There is no public data about git source usage in crates. The number of affected users is unclear. The best I can offer is based on the shared benchmarks

The impact on users with

• a single source (e.g. crates.io) - no impact
• many git dependencies, fresh clone - consistently 2-6x faster cargo update, or even better, depending on the number of git sources and network connection to the server(s).
• many git dependencies, fetching few/no commits - consistently 5x faster (5s-> 1s in the benchmark, 10-20s -> 2-4s for my usual workflow) cargo update
• many git dependencies, fresh shallow clone - 6x faster than serial shallow clone. 9s -> 1.5s in the benchmark, 12x faster than serial full clone

[susitsm]

The comment about blocking

First line

// This isn't strictly necessary since it will be called later.

Most outermost callers of functions returning Poll follow this pattern:

loop {
    match registry_or_source.function(...) {
        Poll::Ready(value) => break value,
        Poll::Pending => registry_or_source.block_until_ready()?,
    }
}

The author most likely meant these block_until_ready call as the will be called later

Rest of the comment

// However it improves error messages for sources that issue errors
// in `block_until_ready` because the callers here have context about
// which deps are being resolved.

The callers most likely mean patch and query. Both have the same snippet:

self.ensure_loaded(dep.source_id(), Kind::Normal)
    .with_context(|| {
        format!(
            "failed to load source for dependency `{}`",
            dep.package_name()
        )
    })?;

The improved error message is these with_context calls that includes the dependency needing the source.

If ensure_loaded is changed without care to return Poll, it will be random which block_until_ready call receives the Poll::Ready(Err(_)) which could result in inconsistent error messages.

[epage]

Thanks for doing those shallow clone benchmarks!

I'm not sure what the expectations are about end user impact. There is no public data about git source usage in crates. The number of affected users is unclear. The best I can offer is based on the shared benchmarks

I brought up end-user impact because benchmarks are just numbers. What matters for evaluating is understanding how this impacts you. You mentioned this is a problem when you are using a lot of git = "..", branch = ".." dependencies and running cargo update. How slow is the cargo update? How many git sources do you have? How frequently are you running it? If very frequent, what is the motivation for doing so?

I want to re-iterate, that I am grateful for your detailed answers and your patience. I am not trying to add busywork for you. The comment highlighted that there are trade offs and so understanding this can help us understand how to weigh those trade offs.

If ensure_loaded is changed without care to return Poll, it will be random which block_until_ready call receives the Poll::Ready(Err(_)) which could result in inconsistent error messages.

So it sounds like you now have an understanding of all of the requirements that this function is trying to manage.

For anyone curious, it looks like the branch for this is https://github.com/susitsm/cargo/tree/pending-source-updates.

I'm going to move this to needing team input to see what others think of the trade offs.

[epage]

We talked about this in today's cargo team meeting.

We appreciate the interest in speeding this up and the work you've put into your prototype so far. We are hesitant about the handling of progress bars and the potential error message quality issues. It might be worthwhile to break this into smaller chunks that can be merged so we can more easily focus on the smaller, more specific design decisions. For example, posting a PR focusing on making GlobalContext Sync. #15676 may even be a smaller milestone to work towards though it helps other workflows and not as much yours.

If you do, some thoughts on making GlobalContext Sync

• Changing RefCell to Mutex
• LazyCell will need to be replaced with something. Not everything we need is stable on OnceLock though some of it could be emulated

[susitsm]

Thanks for the feedback and bringing this up with the team. For me, this is mostly about giving back to the community, my hacked-together branch works just fine for local use. If it ever gets properly implemented, it will surely be multiple PRs. No hurry there.

Making GlobalContext Sync is something I can help with. You mentioned in the internals thread that it will make some other changes a lot easier. I will start with opening a PR for that.

Then later, maybe some small error message improvements in block_until_ready. There might be easy ways to give the same quality of errors there as at the ensure_loaded calls.

Let's meet again at the upcoming PR!