Fix shell commands in bcr testing

[UebelAndre]

No description provided.

[illicitonion]

There's probably scope for tidying this up (e.g. referencing some variables, rather than copy+pasting the lines), but also, I don't think we super need to worry about maintenance here, it's really just Yet Another edit to make on release :D

Thank you, as ever, for all the digging and fixing!

[UebelAndre]

There's probably scope for tidying this up (e.g. referencing some variables, rather than copy+pasting the lines), but also, I don't think we super need to worry about maintenance here, it's really just Yet Another edit to make on release :D

Thank you, as ever, for all the digging and fixing!

Yeah, going through all this I wonder if I shouldn't just call it quits and publish everything within a single rules_rust archive but I think an ideal world is for each extension to be in it's own archive and so I work toward that end. I'll need to document what it takes to add a module and clean all that up. The release.yaml workflow is currently a mess and the infra is otherwise pretty brittle and painful to test. One day though!

[UebelAndre]

I'm struggling to come up with a way to download the new rules_rust archive in CI for --override_module. I'm wondering if I'm hitting some intentional security limitations. @meteorcloudy I'm wondering if you can provide some insight.

[meteorcloudy]

everything within a single rules_rust archive

I think this might be a better idea, with the repository cache, the archive will only be downloaded by Bazel for once.

[UebelAndre]

everything within a single rules_rust archive

I think this might be a better idea, with the repository cache, the archive will only be downloaded by Bazel for once.

Is that such a huge deal with such tiny archives? I think I would prefer the extra download for faster downloads and extracts of just the modules I care about.

[meteorcloudy]

I thought those smaller modules all depend on rules_rust already? Indeed the extract time might be faster.

Anyway, can you explain a bit why you need to download those archives in the presubmit.yml file?

[UebelAndre]

Anyway, can you explain a bit why you need to download those archives in the presubmit.yml file?

Looking at the failure here https://buildkite.com/bazel/bcr-presubmit/builds/9256#0193ac0a-f993-42ec-9d44-747732373250 (bazelbuild/bazel-central-registry#3359), the module is trying to use a directory outside the checkout as rules_rust. I want to have submodules in rules_rust be bzlmod modules backed by their own archives so the MODULE.bazel file needs to work independently. It would seem using local_path_override actually seems to prevent that entirely so I'm trying to instead use --override_module and some gymnastics to download core rules_rust and extract it for use in the bcr tests. Ideally there would be a section in the BCR yaml that lets me pair a module name with a url and does this for me so repositories which publish multiple archives can properly test them without needing to bundle monolithic modules that contain tons of unnecessary/unused stuff by consumers.

[meteorcloudy]

Have you tried archive_override or git_override?

[UebelAndre]

Have you tried archive_override or git_override?

Why is that any different than local_path_override? If I put this in MODULE.bazel then doesn't it impact consumers? If I make it a dev dependency then tests will fail because it can't find the current version. Feels like lose-lose.

[meteorcloudy]

overrides won't propagate to dependent, you can just keep them in the MODULE.bazel file, they will be ignored when the module is used as a dependency. Would that work?

[UebelAndre]

overrides won't propagate to dependent, you can just keep them in the MODULE.bazel file, they will be ignored when the module is used as a dependency. Would that work?

Ah, that's good to know, and that might have worked in combination with some sed hackery. But I instead consolidated everything into a single archive. The round trip of changes to the repo to BCR CI is quite slow and I really wanna move on from this.