publishing: add a CI task to test packaging/building of all published crates individually

[nagisa]

Our current CI tests a combination of all crates in the workspace, which can lead to issues related to crate features especially which are an additive combination. In particular a different crate may be enabling a feature crucial for a crate, meaning the faulty crate would not compile in isolation.

Something like what's in #11983

[mooori]

I'm running into the following problem: We want to dry-run publish crates near-*. Assume this includes:

• near-bar which has no dependency on other packages from near-*
• near-foo which depends on on near-bar

cargo ws publish ... publishes crates in an order that allows satisfying dependencies. Though the point of dry-run is to not actually publish the crates to a registry. Accordingly cargo publish --dry-run -p near-bar succeeds but cargo publish --dry-run -p near-foo fails because near-bar at the new version to be dry-run published cannot be found in the registry.

What if near-bar is a path dependency? My understanding of cargo docs is that this doesn't matter for packaging, which will ignore the path:

Cargo will ignore the path key for dependencies in published packages.

There seems to be no option to enforce using the path dependency. Intuitively this makes sense, as all dependencies of a published crate should be published too.

I can think of the following workarounds:

1. Add --dry-run to NEAR's cargo-workspaces fork and try to implement a workaround there. I doubt this would be practicable as under the hood cargo commands are invoked.
2. Try to use a dummy registry to which we can publish packages during PR CI checks. Not sure of how big of an effort this would be.
3. Publish dummy versions to crates.io that are yanked at the end of the CI run. To me that seems like an abuse of community resources and should therefore be ruled out.

Perhaps I'm missing another workaround that is more promising?

[nagisa]

Dummy registry would probably be ideal approach here as it would emulate the actual workload we're looking to test here the best. There are some projects that provide such a thing, https://github.com/hirevo/alexandrie/ seems most straightforward from a brief look?

List of all sorts of tools are here: https://github.com/rust-lang/cargo/wiki/Third-party-registries.

[mooori]

Thanks for the pointers. I'll give the dummy registry approach a try.

[mooori]

Some further issues popped up after setting up a dummy registry and publishing crates to them:

• Using cargo ws publish --registry ... can publish a crate to the dummy registry, but then it times out because it waits for the crate to become available on crates.io. Even though it was made aware of the alternate registry with --registry. Further investigations are needed to find out if this is an issue with cargo-workspaces upstream or the fork used in nearcore.
  • Another possible root cause could be that the third party registry that I used (kellnr) is sending an incorrect response after the crate was uploaded to it, triggering the unexpected behavior of cargo-workspaces
• Using cargo publish would be possible only after figuring out the correct order in which the near-* crates need to be published since they can depend on each other. cargo workspaces publish can do that, but it has the issue described above.

Making all of this work in CI might require going down some rabbit holes. I'm unassigning myself as I should move on to another project instead.

For anyone who might tackle this in the future, I can recommend kellnr for setting up the dummy registry. It seemed to be the option that's easiest to setup in CI while still providing the required functionality. Heads up that while margo is even easier to set up, it does not (yet) provide the functionality required to cargo publish crates to it. Most software to set up alternate registries seems under active development, so in the future other options might become better suited.