cargo clean --offline fails

[infogulch]

Problem

In some cases cargo clean refuses to execute in offline mode. I can't imagine any valid reason why cargo clean would need access to the network to perform workspace cleaning activity.

Steps

1. Clean up local cargo artifacts
   1. cargo clean
   2. rm -r ~/.cargo/git
   3. rm -r ~/.cargo/registry
2. Change into a directory with a rust project that uses a git dependency (I think this is necessary)
3. Run cargo clean --offline -p <project-name>
   1. This will fail with an error like: error: failed to load source for dependency 'modular-bitfield'

Possible Solution(s)

No response

Notes

No response

Version

cargo 1.75.0-nightly (b4d18d4bd 2023-10-31)
release: 1.75.0-nightly
commit-hash: b4d18d4bd3db6d872892f6c87c51a02999b80802
commit-date: 2023-10-31
host: x86_64-unknown-linux-gnu
libgit2: 1.7.1 (sys:0.18.1 vendored)
libcurl: 8.4.0-DEV (sys:0.4.68+curl-8.4.0 vendored ssl:OpenSSL/1.1.1u)
ssl: OpenSSL 1.1.1u  30 May 2023
os: Ubuntu 22.04 (jammy) [64-bit]

[ehuss]

When using the -p flag, cargo needs to be able to determine which package within the entire dependency tree that the -p flag is referring to. With git dependencies, the only way for cargo to know if the dependency tree is accurate is to refer to the git repository. That in turn requires downloading it. I'm not sure if that is something we can really change or fix, though it might be possible to create a special case for cargo clean --offline, that might be a very intrusive change.

[infogulch]

I see.

I'm running in a CI context where my goal is to clean up packages built for the current workspace but leave dependencies intact so they can be cached; this should minimize cache usage without negatively affecting build times too much since workspace package build outputs are very likely to be invalidated anyway.

My command right now looks like this:

cargo clean -p `cargo metadata --no-deps --offline --format-version 1 | jq -r '[.workspace_members[]|split(" ")|.[0]]|join(" ")'`

Adding a special case for cargo clean --offline would work. But since I'm only interested in cleaning the current workspace packages, not arbitrary packages in the dependency tree, maybe an alternative would be to add a new flag to just clean current workspace packages.

[epage]

Have you looked into the third-party cargo-sweep?

I'm also hoping #5931 will help by having dependency caches separate from caches for local packages. You can then combine that with #12633 which will help clear out per-user cache entries from prior builds.

[weihanglo]

If you have a common prefix for all workspace members, you can use glob syntax like cargo clean -p 'myprefix-*'.

[infogulch]

glob syntax

Yes, but I'm hoping to reuse this same ci script for independent rust projects, so I prefer my general solution despite being a bit longer.

dependency caches separate from caches for local packages

This would be ideal.

The cache is keyed on lock files, host os, and target triple, so "removing stale artifacts" and gc in general is irrelevant in my case (except maybe gc cloned git repos). If deps change it will fail to match the key so deps will be rebuilt, a new cache entry will be created, and the old one will just fall off. Even if it was available today I'd still prefer to rebuild the cache from scratch instead of trying to keep it up to date by gc-ing it.