Please provide a subcommand to refresh the crates.io index

[joshtriplett]

When writing other tools that use the cargo API, and that don't want to unconditionally update the crates.io index from the network, it would help to have a cargo subcommand that refreshes the crates.io index. That avoids having to add such a subcommand to every other such tool built on the cargo API.

[Eh2406]

Or more generally an subcommand for querying the index, so we are not each reinventing it. Maybe move the index stuff into a crate, like the crates.io library. There is rust-crates-index but it is not kept up to date with how cargo maintains the index.

[Eh2406]

Note that crater will in 178 use cargo search lazy_static as a way to get cargo to update the index. Then it will use the -Z no-index-update to skip checking if the index is still up to date.

[Arnavion]

You can use cargo search --limit 0 instead of using a crate like lazy_static.

[nabijaczleweli]

Since 5e680f2, cargo search will no longer update the registry. This has hit those users of cargo install-update who default to nightly so far, but will outright break every installation as that commits gets released down.

This also is a bug in cargo search: it doesn't provide accurate results. We either need a flag for cargo search to always update it, or another command to do so.

[Eh2406]

cc #5621 where the logic for search was changed.
Note that one of the users of the search hack was crater, so when we have somthing that works, they need a PR. In the meantime @pietroalbini how big a deal is it that this is broken?

[pietroalbini]

Uh, this means Crater won't update the registry when generating the lockfiles :/
While cargo update-registry rides the trains I guess we could use cargo install lazy_static, which should do nothing but update the registry (@Eh2406 want to send a PR to Crater?)

[epage]

With the sparse registry, is this command still practical to add?

Note, we also have #11034

[epage]

With this proposal being incompatible with sparse index, I'm closing this. We still have #11034 for the comment about querying the index. If there is a reason we should keep this open, let us know!

[japm48]

@epage

With this proposal being incompatible with sparse index, I'm closing this. We still have #11034 for the comment about querying the index. If there is a reason we should keep this open, let us know!

Could you elaborate on this incompatibility?
I have not read about the sparse index implementation, but I don't understand how there could not be a command that would, at least, check if a local copy of whatever was previously downloaded is outdated.

[epage]

With the git-based registry, the index is downloaded all-or-nothing.

With the sparse registry, each package is kept in an individual file that is downloaded. What to download is context dependent.

You brought up the solution of "re-downloaded whats downloaded". That might work in your case but I suspect it won't work in enough other cases for such a feature to stand on its own. Something like #11034 indirectly offers a way of providing this functionality that adapts to the needs of the user.

[clarfonthey]

In my case, the ability to download the index came from the fact that it was a substantial upfront cost that was useful to perform ahead of time as a cache-warming step. Now that this cost is effectively zero (when downloading dependencies, a majority of the cost is now the dependencies themselves, and not the index), I can rely on just downloading dependencies for crates as I need them.

So, I'm fine with the decision to say, this specific use case isn't necessarily needed, but general index introspection is still good. If you really wanted to download the source for a bunch of crates ahead of time without making a custom empty crate to handle this logic, for example, that would fit under that. Downloading just the index doesn't even solve that problem anyway, since the sources still have to be downloaded separately.