cargo workspace made easier

[soloturn]

Problem

for rolling release linux distributions it would be cool to have a quicker way to rebuild all rust packages. building even small applications can take forever. to give an example, there is cosmic desktop. it consists of 15 or so components: https://github.com/pop-os/cosmic-epoch

the dependencies are mostly the same and built again and again.

Proposed Solution

some flag to cargo would be cool which makes every module build of the same workspace. something like

cargo --workspace todays-workspace

called 15 times should result in reusing everything which was built before. without having a shared lock file, cargo should be intelligent enough to create one by itself.

Notes

No response

[yochananmarqos]

Possible duplicate of #7285.

I would create a cosmic-epoch issue instead. Cargo already seems to be able to handle duplicate dependencies:

https://doc.rust-lang.org/cargo/reference/features.html#feature-resolver-version-2

[soloturn]

i tried now with cargo, and stable. then with nightly, but interrupted it early:

❯ git diff
diff --git a/PKGBUILD b/PKGBUILD
index 9252a4b..88000eb 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -113,7 +113,7 @@ pkgver() {
 prepare() {
   cd cosmic-epoch
   export CARGO_HOME="$srcdir/cargo-home"
-  export RUSTUP_TOOLCHAIN=stable
+  export RUSTUP_TOOLCHAIN=nightly
 
   for submodule in "${_submodules[@]}"; do
     git submodule init "${submodule#*::}"
@@ -144,13 +144,13 @@ prepare() {
 build() {
   cd cosmic-epoch
   export CARGO_HOME="$srcdir/cargo-home"
-  export RUSTUP_TOOLCHAIN=stable
+  export RUSTUP_TOOLCHAIN=nightly
   # note, consider rust build time optimisations: 
   # https://matklad.github.io/2021/09/04/fast-rust-builds.html, 
   # later. for now, ignore warnings, and build with lower priority 
   # to not block user installing this pkg. to speed up build, use "mold" linker, see 
   # https://stackoverflow.com/questions/67511990/how-to-use-the-mold-linker-with-cargo
-  RUSTFLAGS="-A warnings -C link-arg=-fuse-ld=mold"
+  RUSTFLAGS="-A warnings -C link-arg=-fuse-ld=mold -Z threads=3"
   nice just sysext
 }



CARGO_TARGET_DIR=/tmp/crg CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse CARGO_BUILD_JOBS=2 makepkg --syncdeps

and the cargo target is huge:

❯ du -sh
16G	.
❯ find . -name "libiced_core*rlib" -exec ls -la {} \;
2815234 30. Dez 14:32 ./release/deps/libiced_core-ff6b3bea60ba02f3.rlib
3021942 30. Dez 14:03 ./release/deps/libiced_core-53268b753c6d9f0b.rlib
2922220 30. Dez 14:09 ./release/deps/libiced_core-5cb3eda2f0c62f87.rlib
2540418 30. Dez 14:13 ./release/deps/libiced_core-54d1ceda8c98ee86.rlib
2823536 30. Dez 14:24 ./release/deps/libiced_core-d47bd9d7505e923e.rlib
3055982 30. Dez 14:51 ./release/deps/libiced_core-6aa027c7a995df0e.rlib
2281090 30. Dez 15:54 ./release/deps/libiced_core-dd876c734337f2f1.rlib
3056590 30. Dez 14:46 ./release/deps/libiced_core-64f4e2a49f1518bd.rlib
3022126 30. Dez 13:33 ./release/deps/libiced_core-7f5f040f69996f4d.rlib
3056448 30. Dez 14:39 ./release/deps/libiced_core-c43d7c4d38d9029d.rlib

same software, same iced. written many times. that does not make any sense. i would understand 2 times, one for stable, one for nightly.

[Seppel3210]

The issue is that the project is not set up to use a workspace. It's just a repo with multiple git submodules. To be a workspace it would have to have a Cargo.toml in the parent directory with [workspace] ... in it

[weihanglo]

Step back a bit without looked at the proposed solution, this issue is looking for a way toshare build cache without explicitly creating a workspace. We have --target-dir flag and CARGO_TARGET_DIR env today for that purpose, though you might encounter eage cases like build artifact name conflicts. For a better automatic cache sharing, we have #5931 tracking that as well.

If there is a reason you are unable to create a workspace (maybe nested workspace issue #5042?), please share :)

[soloturn]

CARGO_TARGET_DIR is good enough. workspace is an unnecessary complication, you are right @weihanglo . i am even wondering why cargo implemented such a thing.