Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ci): fix earthly ctest #5424

Merged
merged 10 commits into from
Mar 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions barretenberg/cpp/Earthfile
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ preset-wasm:
# wasi-sdk
COPY +get-wasi-sdk/wasi-sdk src/wasi-sdk
DO +RUN_CMAKE --configure="--preset wasm-threads" --build="--target barretenberg.wasm"
RUN ./src/wasi-sdk/bin/llvm-strip ./bin/barretenberg.wasm
RUN ../src/wasi-sdk/bin/llvm-strip ./bin/barretenberg.wasm
SAVE ARTIFACT bin
SAVE IMAGE --cache-hint

Expand Down Expand Up @@ -137,25 +137,27 @@ build: # default target
test:
BUILD +test-clang-format
FROM +preset-release-assert-all
COPY --dir ./srs_db/+build/. srs_db
# Fiddly bit: we need to reconstruct the paths that CMake built with
RUN mkdir -p build && mv bin CMakeCache.txt CMakeFiles build && cd build
COPY --dir ./srs_db/+build/. ../srs_db
RUN GTEST_COLOR=1 ctest -j$(nproc) --output-on-failure

# Functions
RUN_CMAKE:
FUNCTION
# Runs cmake build and leaves binary artifacts at 'bin'. Uses a mounted cache
# for incremental rebuilds.
FUNCTION
ARG configure # cmake configure flags
ARG build # cmake build flags

WORKDIR /build
# Use a mount for incremental builds locally.
# TODO(AD): To be determined: does this get us in trouble in CI?
RUN --mount type=cache,id="$configure-build",target=/build/build \
(cmake $configure -Bbuild || (rm -f build/CMakeCache.txt && cmake $configure -Bbuild)) && \
cmake --build build $build && \
cp -r build/CMakeCache.txt build/CMakeFiles build/bin .
cp -r build build-tmp
# move the temporary build folder back
# this is because the cached build goes away
RUN rm -rf build && mv build-tmp build
WORKDIR /build/build

BENCH_RELEASE:
FUNCTION
Expand Down
9 changes: 4 additions & 5 deletions scripts/earthly-cloud
Original file line number Diff line number Diff line change
Expand Up @@ -60,17 +60,16 @@ if $INTERACTIVE ; then
# don't play nice with tee if interactive
earthly $EARTHLY_FLAGS $@
elif ! earthly $EARTHLY_FLAGS $@ 2>&1 | tee .output.txt >&2 ; then
# we got a failure, handle retries
# we try earthly once, capturing output
# if we get one of our (unfortunate) known failures, handle retries
# TODO potentially handle other intermittent errors here
if grep 'failed to get edge: inconsistent graph state' .output.txt >/dev/null ; then
# TODO we need to limit earthly parallelism or sometimes get
# TODO when earthly is overloaded we sometimes get
# 'failed to solve: failed to get edge: inconsistent graph state'
# so we use arbitrary WAIT statement groups. Pending earthly support.
# for now we just try to recover from the failures.
echo "Got 'inconsistent graph state'. Restarting earthly. See https://github.com/earthly/earthly/issues/2454'"
earthly $EARTHLY_FLAGS $@
else
# otherwise, propagate error
exit 1
fi
fi
fi
Loading