Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce cargo-nightly #29319

Closed
wants to merge 1 commit into from
+37 −27
Closed

introduce cargo-nightly #29319

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions cargo-nightly
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash

# shellcheck source=ci/rust-version.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please put this shellcheck directive directly above source "${here}"/ci/rust-version.sh nightly

here=$(dirname "$0")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a well known(?) pattern used for this:

here=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )

From here: https://stackoverflow.com/a/246128/1989046
7.7k upvotes.

I can see that in other scripts the same here=$(dirname "$0") is used.
So, it is probably out of scope for this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be a good idea to make a separate PR for this: I saw more than one way of doing this in the scripts (ci, scripts folders)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here=$(dirname "$0") is pretty common and it works fine except for cases when a script is expected to be sourced , in which case BASH_SOURCE is certainly needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
here=$(dirname "$0")
here="$(dirname "$0")"


source "${here}"/ci/rust-version.sh nightly
set -x
exec cargo "+${rust_nightly}" "${@}"
21 changes: 11 additions & 10 deletions ci/test-bench.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ source ci/upload-ci-artifact.sh
eval "$(ci/channel-info.sh)"

cargo="$(readlink -f "./cargo")"
cargoNightly="$(readlink -f "./cargo-nightly")"

set -o pipefail
export RUST_BACKTRACE=1
Expand All @@ -32,43 +33,43 @@ test -d target/debug/sbf && find target/debug/sbf -name '*.d' -delete
test -d target/release/sbf && find target/release/sbf -name '*.d' -delete

# Ensure all dependencies are built
_ "$cargo" nightly build --release
_ "$cargoNightly" build --release

# Remove "BENCH_FILE", if it exists so that the following commands can append
rm -f "$BENCH_FILE"

# Run sdk benches
_ "$cargo" nightly bench --manifest-path sdk/Cargo.toml ${V:+--verbose} \
_ "$cargoNightly" bench --manifest-path sdk/Cargo.toml ${V:+--verbose} \
-- -Z unstable-options --format=json | tee -a "$BENCH_FILE"

# Run runtime benches
_ "$cargo" nightly bench --manifest-path runtime/Cargo.toml ${V:+--verbose} \
_ "$cargoNightly" bench --manifest-path runtime/Cargo.toml ${V:+--verbose} \
-- -Z unstable-options --format=json | tee -a "$BENCH_FILE"

# Run gossip benches
_ "$cargo" nightly bench --manifest-path gossip/Cargo.toml ${V:+--verbose} \
_ "$cargoNightly" bench --manifest-path gossip/Cargo.toml ${V:+--verbose} \
-- -Z unstable-options --format=json | tee -a "$BENCH_FILE"

# Run poh benches
_ "$cargo" nightly bench --manifest-path poh/Cargo.toml ${V:+--verbose} \
_ "$cargoNightly" bench --manifest-path poh/Cargo.toml ${V:+--verbose} \
-- -Z unstable-options --format=json | tee -a "$BENCH_FILE"

# Run core benches
_ "$cargo" nightly bench --manifest-path core/Cargo.toml ${V:+--verbose} \
_ "$cargoNightly" bench --manifest-path core/Cargo.toml ${V:+--verbose} \
-- -Z unstable-options --format=json | tee -a "$BENCH_FILE"

# Run sbf benches
_ "$cargo" nightly bench --manifest-path programs/sbf/Cargo.toml ${V:+--verbose} --features=sbf_c \
_ "$cargoNightly" bench --manifest-path programs/sbf/Cargo.toml ${V:+--verbose} --features=sbf_c \
-- -Z unstable-options --format=json --nocapture | tee -a "$BENCH_FILE"

# Run banking/accounts bench. Doesn't require nightly, but use since it is already built.
_ "$cargo" nightly run --release --manifest-path banking-bench/Cargo.toml ${V:+--verbose} | tee -a "$BENCH_FILE"
_ "$cargo" nightly run --release --manifest-path accounts-bench/Cargo.toml ${V:+--verbose} -- --num_accounts 10000 --num_slots 4 | tee -a "$BENCH_FILE"
_ "$cargoNightly" run --release --manifest-path banking-bench/Cargo.toml ${V:+--verbose} | tee -a "$BENCH_FILE"
_ "$cargoNightly" run --release --manifest-path accounts-bench/Cargo.toml ${V:+--verbose} -- --num_accounts 10000 --num_slots 4 | tee -a "$BENCH_FILE"

# `solana-upload-perf` disabled as it can take over 30 minutes to complete for some
# reason
exit 0
_ "$cargo" nightly run --release --package solana-upload-perf \
_ "$cargoNightly" run --release --package solana-upload-perf \
-- "$BENCH_FILE" "$TARGET_BRANCH" "$UPLOAD_METRICS" | tee "$BENCH_ARTIFACT"

upload-ci-artifact "$BENCH_FILE"
Expand Down
5 changes: 3 additions & 2 deletions ci/test-checks.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ source ci/rust-version.sh stable
source ci/rust-version.sh nightly
eval "$(ci/channel-info.sh)"
cargo="$(readlink -f "./cargo")"
cargoNightly="$(readlink -f "./cargo-nightly")"

scripts/increment-cargo-version.sh check

Expand All @@ -34,10 +35,10 @@ echo --- build environment
rustup run "$rust_nightly" rustc --version --verbose

"$cargo" stable --version --verbose
"$cargo" nightly --version --verbose
"$cargoNightly" --version --verbose

"$cargo" stable clippy --version --verbose
"$cargo" nightly clippy --version --verbose
"$cargoNightly" clippy --version --verbose

# audit is done only with "$cargo stable"
"$cargo" stable audit --version
Expand Down
18 changes: 9 additions & 9 deletions scripts/cargo-fmt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
#!/usr/bin/env bash

here="$(dirname "$0")"
cargo="$(readlink -f "${here}/../cargo")"
cargoNightly="$(readlink -f "${here}/../cargo")"

if [[ -z $cargo ]]; then
>&2 echo "Failed to find cargo. Mac readlink doesn't support -f. Consider switching
if [[ -z $cargoNightly ]]; then
>&2 echo "Failed to find cargo-nightly. Mac readlink doesn't support -f. Consider switching
to gnu readlink with 'brew install coreutils' and then symlink greadlink as
/usr/local/bin/readlink."
exit 1
fi

set -ex

"$cargo" nightly fmt --all
(cd programs/sbf && "$cargo" nightly fmt --all)
(cd sdk/cargo-build-sbf/tests/crates/fail && "$cargo" nightly fmt --all)
(cd sdk/cargo-build-sbf/tests/crates/noop && "$cargo" nightly fmt --all)
(cd storage-bigtable/build-proto && "$cargo" nightly fmt --all)
(cd web3.js/test/fixtures/noop-program && "$cargo" nightly fmt --all)
"$cargoNightly" fmt --all
(cd programs/sbf && "$cargoNightly" fmt --all)
(cd sdk/cargo-build-sbf/tests/crates/fail && "$cargoNightly" fmt --all)
(cd sdk/cargo-build-sbf/tests/crates/noop && "$cargoNightly" fmt --all)
(cd storage-bigtable/build-proto && "$cargoNightly" fmt --all)
(cd web3.js/test/fixtures/noop-program && "$cargoNightly" fmt --all)
6 changes: 3 additions & 3 deletions scripts/coverage-in-disk.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ set -e
cd "$(dirname "$0")/.."
source ci/_

cargo="$(readlink -f "./cargo")"
cargoNightly="$(readlink -f "./cargo-nightly")"

: "${CI_COMMIT:=local}"
reportName="lcov-${CI_COMMIT:0:9}"
Expand Down Expand Up @@ -79,8 +79,8 @@ fi
NPROC=$(nproc)
JOBS=$((JOBS>NPROC ? NPROC : JOBS))

RUST_LOG=solana=trace _ "$cargo" nightly test --jobs "$JOBS" --target-dir target/cov --no-run "${packages[@]}"
if RUST_LOG=solana=trace _ "$cargo" nightly test --jobs "$JOBS" --target-dir target/cov "${packages[@]}" 2> target/cov/coverage-stderr.log; then
RUST_LOG=solana=trace _ "$cargoNightly" test --jobs "$JOBS" --target-dir target/cov --no-run "${packages[@]}"
if RUST_LOG=solana=trace _ "$cargoNightly" test --jobs "$JOBS" --target-dir target/cov "${packages[@]}" 2> target/cov/coverage-stderr.log; then
test_status=0
else
test_status=$?
Expand Down
6 changes: 3 additions & 3 deletions scripts/coverage.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ set -e
cd "$(dirname "$0")/.."
source ci/_

cargo="$(readlink -f "./cargo")"
cargoNightly="$(readlink -f "./cargo-nightly")"

: "${CI_COMMIT:=local}"
reportName="lcov-${CI_COMMIT:0:9}"
Expand Down Expand Up @@ -78,8 +78,8 @@ fi
NPROC=$(nproc)
JOBS=$((JOBS>NPROC ? NPROC : JOBS))

RUST_LOG=solana=trace _ "$cargo" nightly test --jobs "$JOBS" --target-dir target/cov --no-run "${packages[@]}"
if RUST_LOG=solana=trace _ "$cargo" nightly test --jobs "$JOBS" --target-dir target/cov "${packages[@]}" -- --nocapture 2> >(tee target/cov/coverage-stderr.log >&2); then
RUST_LOG=solana=trace _ "$cargoNightly" test --jobs "$JOBS" --target-dir target/cov --no-run "${packages[@]}"
if RUST_LOG=solana=trace _ "$cargoNightly" test --jobs "$JOBS" --target-dir target/cov "${packages[@]}" -- --nocapture 2> >(tee target/cov/coverage-stderr.log >&2); then
test_status=0
else
test_status=$?
Expand Down