Build system fails when wrappers for bootstrap binaries are used

[paulmenzel]

When building Rust 1.56.1, specifying our own bootstrap binaries, which are wrappers, in the configuration

[build]
cargo = "/usr/local/package/bin/cargo"
rustc = "/usr/local/package/bin/rustc"
rustfmt = "/usr/local/package/bin/rustfmt"

they are not detected:

$ python3 ./x.py build --exclude src/tools/miri
info: looks like you are running this command under `sudo`
      and so in order to preserve your $HOME this will now
      use vendored sources by default.
error: process didn't exit successfully: `/usr/local/package/bin/rustc -vV` (exit code: 1)
--- stderr
/usr/local/package/bin/rustc : rustc not found

failed to run: /usr/local/package/bin/cargo build --manifest-path /pkg/rustc-1.56.1-0/build/rustc-1.56.1-src/src/bootstrap/Cargo.toml --frozen

But the wrappers work:

$ /usr/local/package/bin/rustc --version
rustc 1.52.1
$ /usr/local/package/bin/cargo --version
cargo 1.52.0
$ more /usr/local/package/bin/rustc
#! /bin/bash
cmd=$(basename "$0")
if [ -n "$PWRAP_rust" ]; then
  echo "$0 : $cmd not found" >&2;
  exit 1
fi
. /usr/local/package/lib/rust.profile
export PWRAP_rust=1
if ! /usr/sbin/hostconfig 'mxqd' ;then
  rcmd=$(/usr/bin/readlink -f $(/usr/bin/which "$cmd"))
  user=$(/usr/bin/id -nu)
  /usr/bin/logger --server log.molgen.mpg.de -t usrlocalpkg "rust.profile ${cmd} (${rcmd}) ${user}@${HOSTNAME} mx64"
fi
exec "$cmd" "$@"

[Mark-Simulacrum]

Does running /usr/local/package/bin/cargo build --manifest-path /pkg/rustc-1.56.1-0/build/rustc-1.56.1-src/src/bootstrap/Cargo.toml --frozen directly work? Initially, it looks likely that this isn't directly related to bootstrap, rather that the wrapper is not finding the binary when invoking it as just rustc on the last line there.

[paulmenzel]

Yes, it does, at least the error is not shown.

$ /usr/local/package/bin/cargo build --manifest-path /pkg/rustc-1.56.1-0/build/rustc-1.56.1-src/src/bootstrap/Cargo.toml --frozen
error: failed to get `cc` as a dependency of package `bootstrap v0.0.0 (/pkg/rustc-1.56.1-0/build/rustc-1.56.1-src/src/bootstrap)`

Caused by:
  attempting to make an HTTP request, but --frozen was specified

[Mark-Simulacrum]

Can you dump the path before running rustc? It's possible that if you remove --frozen you'll see the same error from rustc - I think that we resolve the workspace before rustc invocations.

[paulmenzel]

Can you dump the path before running rustc?

Sure, if you tell me how. ;-)

It's possible that if you remove --frozen you'll see the same error from rustc - I think that we resolve the workspace before rustc invocations.

No, works as expected.

$ /usr/local/package/bin/cargo build --manifest-path /pkg/rustc-1.56.1-0/build/rustc-1.56.1-src/src/bootstrap/Cargo.toml
    Updating crates.io index
[…]

[Mark-Simulacrum]

Sure, if you tell me how. ;-)

Ultimately, I don't have an exact sequence of steps to offer -- it seems likely that there is some difference in the environment between the direct invocation and the one down within bootstrap. But what that difference is requires some level of debugging -- it could be PATH differences (but seems somewhat unlikely), it may be that the wrapper(s) fail when some of the env variables bootstrap sets are set (e.g., RUSTC).

/usr/local/package/bin/rustc : rustc not found looks like a somewhat atypical message, in the sense that I don't think it's something we're emitting -- identifying why the rustc there is not found is probably the first step. Without more details on the wrappers you have it's hard for me to know why this might happen though.

[paulmenzel]

It is from our wrapper:

if [ -n "$PWRAP_rust" ]; then
  echo "$0 : $cmd not found" >&2;
  exit 1
fi

I believe, it’s some kind protection against calling the wrapper infinitely often, but I am going to ask my colleague.

[Mark-Simulacrum]

My suspicion is that the execution of rustc without an absolute path in the last line of the wrapper is problematic; it may be useful to get a trace of what commands are run, but generally executing a bare rustc from the environment seems error-prone (it is easy for that to use the wrong rustc).

[paulmenzel]

No, that is not it, as the wrapper correctly sets the paths. The wrapper does not allow to be called recursively, and that is the check I referenced. We think about introducing a counter to allow it to be called recursively a limited amount of time.

[Mark-Simulacrum]

I don't think we'd expect it to be called recursively though? At least, I don't see a reason off hand bootstrap would do that.

[paulmenzel]

cargo and rust are from the same “package” rust, and, therefore, use the same wrapper. And rustc seems to be called by cargo?

[Mark-Simulacrum]

Oh. I didn't realize they all used the same wrapper/recursive detection. I don't really understand why that ever worked though (bootstrap does nothing special; Cargo always calls rustc...).

[paulmenzel]

I don't really understand why that ever worked though (bootstrap does nothing special; Cargo always calls rustc...).

In the past we didn’t manually specify the paths to rustc, cargo and rustfmt but the src/stage0.json versions were downloaded, As we want to build as much from source as possible, we tried to use the already installed version, and ran into the problems.

[paulmenzel]

Thank you for your help!