RFC: resolve crates.io source replacement ambiguity #3289

arlosi · 2022-07-05T19:27:09Z

When Cargo is performing an API operation (yank/login/publish/etc.) to a source-replaced crates-io, require the user to pass --registry <NAME> to specify exactly which registry to use. Additionally, ensure that the token for crates-io is never sent to a replacement registry.

Rendered

jmyersmsft · 2022-07-07T19:53:52Z

I'd expand the language in Change 3 to clarify that credentials for one registry must not be sent to any other registry, regardless of whether either is crates.io. Any exception to that should have to be an explicit configuration by the user (and probably not in a place that would come from source control), or something controlled by the registry for which you have credentials (e.g. maybe crates.io could somehow say official-second-site-of-crates-io.net can receive crates.io credentials)

…aced API operations an error

ehuss · 2022-07-09T01:09:16Z

text/0000-source_replacement_ambiguity.md

+
+Should the `--registry <NAME>` command line argument be allowed to reference the name of a `source` from the `[source]` table as well? This makes it more flexible, but adds potentially unnecessary complexity.
+
+Cargo's tests rely on the ability to replace the crates.io source and have the crates.io credentials go to the replaced source. We need a way for these tests to continue working. 


I'm not too worried about this, as I believe all of the relevant tests could be adjusted to include --registry if necessary. In the past, the tests would pass --index, though that was removed in rust-lang/cargo#7973.

It is unfortunate that it is difficult to test cargo's behavior without using source replacement. There are some tests that are impossible to write, but I don't think it should be too much of a problem.

ehuss · 2022-07-12T23:58:40Z

Thanks @arlosi for putting this together! This seems reasonable to me, so I'm going to go ahead and start FCP. We've been discussing this off and on for quite a while now, and this I believe matches what we all agree on.

@rfcbot fcp merge

rfcbot · 2022-07-12T23:58:41Z

Team member @ehuss has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

ehuss · 2022-07-13T00:06:49Z

Oh, I did just think of one thing. If you are using a mirror of crates-io, and you run cargo publish --registry crates-io, I believe this RFC is indicating that the verification step will use the index directly from crates-io ("Cargo ignores source replacement when building and publishing the crate to crates.io."). However, I believe there are scenarios where that won't work (or won't work well). For example, I believe many users in China are using the mirror from USTC, since otherwise there are network problems.

I'm wondering if it would make sense to leave the verification step to use source replacement? I can't think of a strong reason to change that. The mirror might lag the original source, but in that case the user can either wait if they are publishing multiple inter-dependent packages, or they can use --no-verify.

Can you say more why that particular statement is there?

arlosi · 2022-07-13T03:46:18Z

Yes, when using --registry crates-io all steps would be performed with crates.io directly.

If the verification step followed source replacement, then Cargo would need to fetch the index for the replacement and crates.io (the api URL is stored in config.json in the index, so we need to fetch that from the real crates.io). It seemed better to avoid fetching two indexes, and easier to explain.

It's currently not possible to publish to crates.io with source replacement configured, so this is not a loss of functionality. Users in China currently have to disable the USTC replacement in the config in order to publish.

I suppose if we were using -Z sparse-registry, then we could fetch only config.json from crates.io and the rest of the index from the mirror? Or maybe it's not a big deal to fetch two indexes?

Edit: In the implementation of this RFC, the verification build does use the source replacement.

rfcbot · 2022-07-13T16:18:44Z

🔔 This is now entering its final comment period, as per the review above. 🔔

weihanglo

Thank you for writing this! Just some nitpicks.

text/0000-source_replacement_ambiguity.md

Co-authored-by: Weihang Lo <weihanglo@users.noreply.github.com>

rfcbot · 2022-07-23T16:34:11Z

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

ehuss · 2022-07-23T23:07:16Z

Huzzah! The @rust-lang/cargo team has decided to accept this RFC.

To track further discussion, subscribe to the tracking issue here:
rust-lang/cargo#10894

Implement RFC 3289: source replacement ambiguity ### Implements [RFC 3289](rust-lang/rfcs#3289) * When the crates-io source is replaced, the user needs to specify `--registry <NAME>` when running an API operation to disambiguate which registry to use. Otherwise, cargo will issue a new error. * In source replacement, the `replace-with` key can reference the name of an alt registry in the `[registries]` table. * Publishing to source-replaced crates.io is no longer permitted using the crates.io token (`registry.token`). We have had a deprecation warning in place since #7973 (1.45.0). ### Testing * Tests are updated to add the `--registry dummy-registry` parameter to specify the test registry (otherwise they would get the new error message) * A few tests that need to verify crates-io-specific configuration use an internal `allow_silent_crates_io_replacement` function to allow the previous behavior of silently replacing crates.io within the testing framework. Changes are insta-stable. cc #10894 r? `@Eh2406`

Implement RFC 3289: source replacement ambiguity ### Implements [RFC 3289](rust-lang/rfcs#3289) * When the crates-io source is replaced, the user needs to specify `--registry <NAME>` when running an API operation to disambiguate which registry to use. Otherwise, cargo will issue a new error. * In source replacement, the `replace-with` key can reference the name of an alt registry in the `[registries]` table. * Publishing to source-replaced crates.io is no longer permitted using the crates.io token (`registry.token`). We have had a deprecation warning in place since #7973 (1.45.0). ### Testing * Tests that interacting with crates.io use the new `replace_crates_io` function, which internally sets an environment variable to change the URL of crates.io. Changes are insta-stable. cc #10894 r? `@Eh2406`

arlosi changed the title ~~RFC: source_replacement_ambiguity~~ RFC: resolve crates.io source replacement ambiguity Jul 5, 2022

Add RFC source_replacement_ambiguity

502444c

arlosi force-pushed the sr branch from 4e01a4f to 502444c Compare July 5, 2022 19:28

ehuss added the T-cargo Relevant to the Cargo team, which will review and decide on the RFC. label Jul 6, 2022

arlosi mentioned this pull request Jul 7, 2022

Fix publishing to crates.io with -Z sparse-registry rust-lang/cargo#10831

Merged

Add details about Change 3; Add additional alternative of making repl…

a65694a

…aced API operations an error

ehuss reviewed Jul 9, 2022

View reviewed changes

rfcbot added proposed-final-comment-period Currently awaiting signoff of all team members in order to enter the final comment period. disposition-merge This RFC is in PFCP or FCP with a disposition to merge it. labels Jul 12, 2022

rfcbot added final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. and removed proposed-final-comment-period Currently awaiting signoff of all team members in order to enter the final comment period. labels Jul 13, 2022

weihanglo reviewed Jul 19, 2022

View reviewed changes

arlosi and others added 2 commits July 22, 2022 08:49

Apply suggestions from code review

8f1c4bc

Co-authored-by: Weihang Lo <weihanglo@users.noreply.github.com>

Add RFC hyperlinks

f77f7df

rfcbot added finished-final-comment-period The final comment period is finished for this RFC. and removed final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. labels Jul 23, 2022

rfcbot added the to-announce label Jul 23, 2022

ehuss mentioned this pull request Jul 23, 2022

Tracking Issue for source replacement ambiguity rust-lang/cargo#10894

Closed

1 task

Update tracking issue

894b770

ehuss merged commit 2609dc0 into rust-lang:master Jul 23, 2022

This was referenced Jul 27, 2022

Implement RFC 3139: alternative registry authentication support rust-lang/cargo#10592

Merged

Implement RFC 3289: source replacement ambiguity rust-lang/cargo#10907

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RFC: resolve crates.io source replacement ambiguity #3289

RFC: resolve crates.io source replacement ambiguity #3289

arlosi commented Jul 5, 2022 •

edited by ehuss

Loading

jmyersmsft commented Jul 7, 2022 •

edited

Loading

ehuss Jul 9, 2022

ehuss commented Jul 12, 2022

rfcbot commented Jul 12, 2022 •

edited by weihanglo

Loading

ehuss commented Jul 13, 2022

arlosi commented Jul 13, 2022 •

edited

Loading

rfcbot commented Jul 13, 2022

weihanglo left a comment

rfcbot commented Jul 23, 2022

ehuss commented Jul 23, 2022


		Should the `--registry <NAME>` command line argument be allowed to reference the name of a `source` from the `[source]` table as well? This makes it more flexible, but adds potentially unnecessary complexity.

		Cargo's tests rely on the ability to replace the crates.io source and have the crates.io credentials go to the replaced source. We need a way for these tests to continue working.

RFC: resolve crates.io source replacement ambiguity #3289

RFC: resolve crates.io source replacement ambiguity #3289

Conversation

arlosi commented Jul 5, 2022 • edited by ehuss Loading

jmyersmsft commented Jul 7, 2022 • edited Loading

ehuss Jul 9, 2022

Choose a reason for hiding this comment

ehuss commented Jul 12, 2022

rfcbot commented Jul 12, 2022 • edited by weihanglo Loading

ehuss commented Jul 13, 2022

arlosi commented Jul 13, 2022 • edited Loading

rfcbot commented Jul 13, 2022

weihanglo left a comment

Choose a reason for hiding this comment

rfcbot commented Jul 23, 2022

ehuss commented Jul 23, 2022

arlosi commented Jul 5, 2022 •

edited by ehuss

Loading

jmyersmsft commented Jul 7, 2022 •

edited

Loading

rfcbot commented Jul 12, 2022 •

edited by weihanglo

Loading

arlosi commented Jul 13, 2022 •

edited

Loading