Skip to content

./miri improvements #2296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bors merged 3 commits into from
Jul 1, 2022
Merged

./miri improvements #2296

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a1fabb9
make miri script work from other working directories
RalfJung Jul 1, 2022
7d09313
add './miri many-seeds', and respect MIRIFLAGS in './miri run'
RalfJung Jul 1, 2022
9bc7938
more tweaks
RalfJung Jul 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 68 additions & 46 deletions miri
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,31 +5,37 @@ USAGE=$(cat <<"EOF"

./miri install <flags>:
Installs the miri driver and cargo-miri. <flags> are passed to `cargo
install`. Sets up the rpath such that the installed binary should work in any
install`. Sets up the rpath such that the installed binary should work in any
working directory.

./miri build <flags>:
Just build miri. <flags> are passed to `cargo build`.
Just build miri. <flags> are passed to `cargo build`.

./miri check <flags>:
Just check miri. <flags> are passed to `cargo check`.
Just check miri. <flags> are passed to `cargo check`.

./miri test <flags>:
Build miri, set up a sysroot and then run the test suite. <flags> are passed
to the final `cargo test` invocation.

./miri run <flags>:
Build miri, set up a sysroot and then run the driver with the given <flags>.
(Also respects MIRIFLAGS environment variable.)

The commands above also exist in a "-debug" variant (e.g. "./miri run-debug
<flags>") which uses debug builds instead of release builds, for faster build
times and slower execution times.

./miri fmt <flags>:
Format all sources and tests. <flags> are passed to `rustfmt`.
Format all sources and tests. <flags> are passed to `rustfmt`.

./miri clippy <flags>:
Format all sources and tests. <flags> are passed to `cargo clippy`.
Format all sources and tests. <flags> are passed to `cargo clippy`.

./miri many-seeds <command>:
Runs <command> over and over again with different seeds for Miri. The MIRIFLAGS
variable is set to its original value appended with ` -Zmiri-seed=$SEED` for
many different seeds.

ENVIRONMENT VARIABLES

Expand All @@ -41,18 +47,39 @@ Pass extra flags to all cargo invocations.
EOF
)

# Determine command.
COMMAND="$1"
[ $# -gt 0 ] && shift

## Handle some commands early, since they should *not* alter the environment.
case "$COMMAND" in
many-seeds)
Copy link
Contributor

@oli-obk oli-obk Jul 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what on earth is this syntax. I have no idea how bash works XD gonna have to trust you here

Copy link
Member Author

@RalfJung RalfJung Jul 1, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😂 it's the same syntax we already use elsewhere ;) Think match COMMAND { "many-seeds" => ... }

Copy link
Contributor

@oli-obk oli-obk Jul 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea I assumed that's what it is, but it looks so weird.

for SEED in $({ echo obase=16; seq 0 255; } | bc); do
MIRIFLAGS="$MIRIFLAGS -Zmiri-seed=$SEED" $@ || { echo "Failing seed: $SEED"; break; }
done
exit 0
;;
esac

## Preparation
TARGET=$(rustc --version --verbose | grep "^host:" | cut -d ' ' -f 2)
SYSROOT=$(rustc --print sysroot)
LIBDIR=$SYSROOT/lib/rustlib/$TARGET/lib
# macOS does not have a useful readlink/realpath so we have to use Python instead...
MIRIDIR=$(dirname "$(python3 -c 'import os, sys; print(os.path.realpath(sys.argv[1]))' "$0")")
MIRIDIR=$(python3 -c 'import os, sys; print(os.path.dirname(os.path.realpath(sys.argv[1])))' "$0")
# Determine toolchain *in the Miri dir* and use that.
TOOLCHAIN=$(cd "$MIRIDIR"; rustup show active-toolchain | head -n 1 | cut -d ' ' -f 1)

# Determine some toolchain properties
TARGET=$(rustc +$TOOLCHAIN --version --verbose | grep "^host:" | cut -d ' ' -f 2)
SYSROOT=$(rustc +$TOOLCHAIN --print sysroot)
LIBDIR=$SYSROOT/lib/rustlib/$TARGET/lib
if ! test -d "$LIBDIR"; then
echo "Something went wrong determining the library dir."
echo "I got $LIBDIR but that does not exist."
echo "Please report a bug at https://github.com/rust-lang/miri/issues."
exit 2
fi

# Prepare flags for cargo and rustc.
CARGO="cargo +$TOOLCHAIN"
if [ -z "$CARGO_INCREMENTAL" ]; then
# Default CARGO_INCREMENTAL to 1.
export CARGO_INCREMENTAL=1
Expand All @@ -65,18 +92,31 @@ fi
# We enable debug-assertions to get tracing.
# We enable line-only debuginfo for backtraces.
export RUSTFLAGS="-C link-args=-Wl,-rpath,$LIBDIR -C debug-assertions -C debuginfo=1 $RUSTFLAGS"
# Determine flags passed to all cargo invocations.
# This is a bit more annoying that one would hope due to
# <https://github.com/rust-lang/cargo/issues/6992>.
case "$COMMAND" in
*-debug)
CARGO_INSTALL_FLAGS="--target $TARGET --debug $CARGO_EXTRA_FLAGS"
CARGO_BUILD_FLAGS="--target $TARGET $CARGO_EXTRA_FLAGS"
;;
*)
CARGO_INSTALL_FLAGS="--target $TARGET $CARGO_EXTRA_FLAGS"
CARGO_BUILD_FLAGS="--target $TARGET --release $CARGO_EXTRA_FLAGS"
;;
esac

## Helper functions

# Build a sysroot and set MIRI_SYSROOT to use it. Arguments are passed to `cargo miri setup`.
# Build a sysroot and set MIRI_SYSROOT to use it. Arguments are passed to `cargo miri setup`.
build_sysroot() {
# Build once, for the user to see.
cargo run $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/cargo-miri/Cargo.toml -- miri setup "$@"
$CARGO run $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/cargo-miri/Cargo.toml -- miri setup "$@"
# Call again, to just set env var.
export MIRI_SYSROOT="$(cargo run $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/cargo-miri/Cargo.toml -q -- miri setup --print-sysroot "$@")"
export MIRI_SYSROOT="$($CARGO run $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/cargo-miri/Cargo.toml -q -- miri setup --print-sysroot "$@")"
}

# Prepare and set MIRI_SYSROOT. Respects `MIRI_TEST_TARGET` and takes into account
# Prepare and set MIRI_SYSROOT. Respects `MIRI_TEST_TARGET` and takes into account
# locally built vs. distributed rustc.
find_sysroot() {
if [ -n "$MIRI_SYSROOT" ]; then
Expand All @@ -93,45 +133,27 @@ find_sysroot() {

## Main

# Determine command.
COMMAND="$1"
[ $# -gt 0 ] && shift

# Determine flags passed to all cargo invocations.
# This is a bit more annoying that one would hope due to
# <https://github.com/rust-lang/cargo/issues/6992>.
case "$COMMAND" in
*-debug)
CARGO_INSTALL_FLAGS="--target $TARGET --debug $CARGO_EXTRA_FLAGS"
CARGO_BUILD_FLAGS="--target $TARGET $CARGO_EXTRA_FLAGS"
;;
*)
CARGO_INSTALL_FLAGS="--target $TARGET $CARGO_EXTRA_FLAGS"
CARGO_BUILD_FLAGS="--target $TARGET --release $CARGO_EXTRA_FLAGS"
;;
esac

# Run command.
case "$COMMAND" in
install|install-debug)
# "--locked" to respect the Cargo.lock file if it exists,
# "--offline" to avoid querying the registry (for yanked packages).
cargo install $CARGO_INSTALL_FLAGS --path "$MIRIDIR" --force --locked --offline "$@"
cargo install $CARGO_INSTALL_FLAGS --path "$MIRIDIR"/cargo-miri --force --locked --offline "$@"
$CARGO install $CARGO_INSTALL_FLAGS --path "$MIRIDIR" --force --locked --offline "$@"
$CARGO install $CARGO_INSTALL_FLAGS --path "$MIRIDIR"/cargo-miri --force --locked --offline "$@"
;;
check|check-debug)
# Check, and let caller control flags.
cargo check $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/Cargo.toml --all-targets "$@"
cargo check $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/cargo-miri/Cargo.toml "$@"
$CARGO check $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/Cargo.toml --all-targets "$@"
$CARGO check $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/cargo-miri/Cargo.toml "$@"
;;
build|build-debug)
# Build, and let caller control flags.
cargo build $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/Cargo.toml "$@"
cargo build $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/cargo-miri/Cargo.toml "$@"
$CARGO build $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/Cargo.toml "$@"
$CARGO build $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/cargo-miri/Cargo.toml "$@"
;;
test|test-debug|bless|bless-debug)
# First build and get a sysroot.
cargo build $CARGO_BUILD_FLAGS
$CARGO build $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/Cargo.toml
find_sysroot
case "$COMMAND" in
bless|bless-debug)
Expand All @@ -140,8 +162,8 @@ test|test-debug|bless|bless-debug)
esac
# Then test, and let caller control flags.
# Only in root project and ui_test as `cargo-miri` has no tests.
cargo test $CARGO_BUILD_FLAGS "$@"
cargo test $CARGO_BUILD_FLAGS --manifest-path ui_test/Cargo.toml "$@"
$CARGO test $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/Cargo.toml "$@"
$CARGO test $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/ui_test/Cargo.toml "$@"
;;
run|run-debug)
# Scan for "--target" to set the "MIRI_TEST_TARGET" env var so
Expand All @@ -157,19 +179,19 @@ run|run-debug)
done
fi
# First build and get a sysroot.
cargo build $CARGO_BUILD_FLAGS
$CARGO build $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/Cargo.toml
find_sysroot
# Then run the actual command.
exec cargo run $CARGO_BUILD_FLAGS -- --sysroot "$MIRI_SYSROOT" "$@"
exec $CARGO run $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/Cargo.toml -- --sysroot "$MIRI_SYSROOT" $MIRIFLAGS "$@"
;;
fmt)
find "$MIRIDIR" -not \( -name target -prune \) -name '*.rs' \
| xargs rustfmt --edition=2021 --config-path "$MIRIDIR/rustfmt.toml" "$@"
| xargs rustfmt +$TOOLCHAIN --edition=2021 --config-path "$MIRIDIR/rustfmt.toml" "$@"
;;
clippy)
cargo clippy $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/Cargo.toml --all-targets "$@"
cargo clippy $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/ui_test/Cargo.toml --all-targets "$@"
cargo clippy $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/cargo-miri/Cargo.toml "$@"
$CARGO clippy $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/Cargo.toml --all-targets "$@"
$CARGO clippy $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/ui_test/Cargo.toml --all-targets "$@"
$CARGO clippy $CARGO_BUILD_FLAGS --manifest-path "$MIRIDIR"/cargo-miri/Cargo.toml "$@"
;;
*)
if [ -n "$COMMAND" ]; then
Expand Down