Reduce the need for users to write build scripts

[epage]

Like RUSTFLAGS, build scripts are an important escape hatch. Like RUSTFLAGS (#12739), we should find replacements for common uses of build scripts so people don't have to reach to this escape hatch so often

Reducing build scripts would

• Improve build times
• Reduce risk of bugs
• Reduce the dependency review audit scope

Uses of build scripts

• Version detection
  • Stabilizing Tracking issue for RFC 2523, #[cfg(accessible(::path::to::thing))] rust#64797
  • Stabilizing Tracking issue for RFC 2523, #[cfg(version(..))] rust#64796
• Making cfg values available at runtime (e.g. in --bugreport, --version, or crash reports, or for tests building examples)
  • Possibly provide cfg_value!, cfg_values!, try_cfg_value!, try_cfg_values! (see also mutually exclusive features)
• Making TARGET available at runtime (e.g. --version, building examples in tests)
• Codegen
  • using tests
  • Allow applying static code-generation to the published package (i.e. package.rs) #12552
• cfg_aliases (see also Reduce the need for users to write build scripts #14948 (comment))
• Catch-all: Tracking issue for RFC 2196, "metabuild: semantic build scripts for Cargo" #14903
  • Note: this would also allow consolidating the ecosystem which would help with baking-in more features to replace build scripts
  • Enable linker warnings on Windows
  • Embedded windows manifests
  • Can we use this for -sys crates / FFI?

[ChrisDenton]

I would draw a distinction between main binary build scripts (e.g. the executable or cdylib) and libraries. I think a build scripts for an executable is far more tolerable than for libraries. They're in direct control of the crate author and don't have any other users. Also a single top-level build script is going to have negligible impact on build times for all but the smallest projects.

Which is not to say reducing the need for them is not important too, just that reducing the need for library build scripts will likely have the far bigger impact.

[epage]

This is not about trying to remove all build scripts but finding patterns in the community and providing a solution to avoid writing one yourself. In some cases, this will be using an artifact dependency as your build script. This is a general solution that helps libs and bins. And the benefits extend to bins. For smaller projects it democratizes the benefits of build scripts by making it easier to discover what you can do, easier to implement, and with fewer bugs. For larger projects with multiple bins, it makes reuse easier, reduces build times, and makes it easier to audit your own code base.

Yes, libs will get more benefits as we prioritize solutions to implement but understanding and tracking the needs of bins takes little away from that but instead gives a better understanding for designing soluations.

[nicoburns]

Can we add "cfg aliases" to this list (i.e. what is implemented using a build script by the https://docs.rs/cfg_aliases/latest/cfg_aliases/ crate)?

Winit is a prominent crate that depends on this cfg_aliases, and from a Dioxus/Blitz perspective I understand why they've done this. We support 6 platforms (windows/macos/linux/ios/android/web) and try to provide fine(ish)-grained feature flagging for each supported feature to allow users to optimise build times and binary sizes. This leads to cfg expressions that spread to 6 lines or more.

Use cases where this is commonly comes up are where there is functionality that works on N of M platforms (e.g. 3 of 6), that you also want to be controllable with a feature flag (users may or may not want the function), and that you want to be disabled on platforms where it doesn't work even when the feature is enabled (as otherwise it becomes burdensome for users to set platform-specific features (where this is even possible)).

To give an entirely concrete example. In blitz-shell we use muda for implementing system menus on windows/macos/linux(desktop), but this functionality isn't available and doesn't make sense on android/ios/web. We also wish to make it optional on platforms that do support it. So we need a feature flag AND platform-specific cfgs, which ends up being a lot of boilerplate.

It would be nice if we could define aliases (either in lib.rs or Cargo.toml) and then use a simple #[cfg(menu)].

[epage]

@nicoburns I've added cfg_aliases. An intermediate solution would be to stabilize metadeps and then you could declare cfg_aliases as one of your build scripts, passing parameters to it. This would mean only one build script bin related to cfg_alises would be built across your entire dependency tree or need to be audited.

[epage]

I feel like the use cases given might have some overlap with global, mutually exclusive features and I wonder if there is a way to fit a higher level construct into that.

[heaths]

Another use case we have is to record the rustc version used to compile with our library that we telemeter, if enabled. Currently I don't see anything obvious recorded in ELF or PE/COFF executables that record the rustc version used - perhaps it's in the debug info, but that may not have been deployed - but perhaps making some basic info available in header or section data - perhaps the debug section - could be exposed at runtime. I admit I don't know much about ELF - Windows had been my primary dev env for decades - but looking at a release-build from rustc, it does look like GCC is recording quite a bit of build/version information so maybe it's warranted to add some rustc info? MSVC (and apparently clang on Windows now) also stores some linker info in the PE/COFF as well.

I mean, there's other ways as well like linking in function to get info, but stamping this in the executable might be useful to external tools as well. Just some thoughts.

[bjorn3]

Currently I don't see anything obvious recorded in ELF or PE/COFF executables that record the rustc version used

The .comment section for ELF executables contains the version of rustc as well as many other tools that were involved in producing the executable. For example:

$ readelf --string-dump=.comment ~/.cargo/bin/rustup

String dump of section '.comment':
  [     0]  GCC: (GNU) 4.8.5 20150623 (Red Hat 4.8.5-44)
  [    2d]  GCC: (GNU) 9.5.0
  [    3e]  rustc version 1.77.2 (25ef9e3d8 2024-04-09)
  [    6a]  clang version 18.1.0rc

As for PE/COFF executables, it looks like the PDB debuginfo file will contain the rustc string. At least https://rust.godbolt.org/z/3zxrKehsK shows the clang LLVM (rustc version 1.84.0 (9fc6b4312 2025-01-07)) string ending up in the .debug$S section of the object file.

[heaths]

Interesting. I ran readelf -a <path> and it didn't dump the .comment section, but doing what you did explicitly worked. Maybe std could expose that if there's enough demand.

As for PE/COFF, I expected this was in the .pdb. somewhere but it's rare symbols are deployed. What might be ideal is to actually store it in a debug directory like MSVC does, something to the effect of:

$ dumpbin /headers /section:.rdata <exe>

# ...
  Debug Directories

        Time Type        Size      RVA  Pointer
    -------- ------- -------- -------- --------
    678FF0B6 cv            4E 00003960     2960    Format: RSDS, {0466A304-9921-46F8-8636-239DB83AA58C}, 1, <path>.pdb
    678FF0B6 feat          14 000039B0     29B0    Counts: Pre-VC++ 11.00=0, C/C++=35, /GS=35, /sdl=1, guardN=34
    678FF0B6 coffgrp      30C 000039C4     29C4    4C544347 (LTCG)
    678FF0B6 iltcg          0 00000000        0

[LeonMatthesKDAB]

Another important use-case I want to mention is FFI (especially with C/C++), which probably broadly fits under the codegen category.

build scripts are encouraged for bindgen, CXX and CXX-Qt.
Where CXX and CXX-Qt both rely on the cc crate, which also encourages use in build scripts.

Especially in the case of cc/CXX/CXX-Qt, this means the build script invokes the C/C++ compiler.
This can be quite detrimental to compile time, as the ever-changing out dir requires a total rebuild of all C++ code even on simple changes.
Incremental builds are currently impossible for the C++ code.

Here are the timing results of rebuilding CXX-Qt after touching a single file in cxx-qt-gen:
cargo-timing.zip

In this example, the build scripts take up about 17.34s out of 20.5s (~85%).

[epage]

@LeonMatthesKDAB FFI was listed under #14903 in the list. While maybe we can eventually find a stable interface to declare these (though I suspect we won't), sharing more of the implementation of -sys crates would help reduce bugs and reduce the number of host binaries that need to be built.

As for problems with an ever-changing out dir, I'm not seeing an existing issue and it might be good to open one for us to explore what could be done. Granted, if we could find a way to have C/C++ be a part of our build graph and know how to handle intermediate files, that would be amazing. However, that is probably even farther in the future and builds on the work of #14903