Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardised verifiable builds #1148

Merged
merged 73 commits into from
Jul 13, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
b3eafd3
docker image + tokio
Jun 5, 2023
a11bb5b
docker client draft
Jun 5, 2023
ade7003
refactoring execute func
Jun 6, 2023
26328c6
successful docker build
Jun 6, 2023
773a2ff
persist flags in the main execution context
Jun 6, 2023
763cc77
docker readme + respect host flags
Jun 7, 2023
300b725
add build steps
Jun 7, 2023
7dca32b
better image, custom image arg, container reuse
Jun 8, 2023
55b72f1
fix readme example
Jun 8, 2023
f63a53e
Merge branch 'master' into gn/verifiable-build
Jun 8, 2023
9955f25
fix docs
Jun 8, 2023
5fe03fb
update README
Jun 8, 2023
47d63d0
docs fixes
Jun 12, 2023
6d3d7fd
clarification in the docs
Jun 12, 2023
334ed7c
use locked for cargo install
Jun 12, 2023
55ce730
concat install and cleanup commands
Jun 12, 2023
54e0291
use arg instead of env in dockerfile
Jun 13, 2023
e28ddcb
add version labels
Jun 13, 2023
a51ea03
update comments
Jun 13, 2023
80c9252
decompose the function
Jun 13, 2023
fd4773a
correct argument passing in dockerfile
Jun 13, 2023
b2eb769
refactoring
Jun 13, 2023
23bd332
more fixes
Jun 13, 2023
5bb5ca4
Merge branch 'master' into gn/verifiable-build
Jun 13, 2023
c706c83
using remote registry + bell & whistles (progress bars)
Jun 13, 2023
4c69b28
changelog entry
Jun 13, 2023
427f27b
fixes
Jun 13, 2023
000d302
refactoring
Jun 14, 2023
a8db873
explicitly specify versions of apt packages
Jun 15, 2023
8264529
refactoring
Jun 20, 2023
a86dcce
add module docs
Jun 20, 2023
5722084
Merge branch 'master' into gn/verifiable-build
Jun 20, 2023
2c419eb
refactoring
Jun 20, 2023
085a744
doc fix and optionally use git in docker image
Jun 20, 2023
913d2a9
Error handling for builds
Jun 20, 2023
1d84b3c
use single var in docker and refactor docker module
Jun 21, 2023
d237d73
notes on apple silicon
Jun 21, 2023
d0e00dd
remove unused arg in dockerfile
Jun 21, 2023
2f35698
use target path correctly
Jun 21, 2023
3e157f4
handle building with relative paths + permissions
Jun 26, 2023
b78d91e
Merge branch 'master' into gn/verifiable-build
Jun 26, 2023
9800402
more flags in docker image + fixes
Jun 26, 2023
e489c74
Merge branch 'master' into gn/verifiable-build
Jun 26, 2023
2fd00cf
remove unused code
Jun 26, 2023
02aa0af
add git info to docker metadata
Jun 26, 2023
fa95d79
use env to detect running OS
Jun 26, 2023
ea251da
detect host OS
Jun 26, 2023
696a5d1
non-root docker user
Jun 28, 2023
acf89ad
mount contract to home dir in docker
Jun 28, 2023
217ab6a
give permission to target to everyone
Jun 28, 2023
707bfb1
fix typo
Jun 28, 2023
f620dcc
use current uid and gid for docker
Jun 29, 2023
37ae55d
conditional compilation
Jun 29, 2023
dd6a89d
dont cache ci template and use musl in docker
Jun 29, 2023
5ec303b
OS specific code blocks
Jun 29, 2023
7209fa3
print error logs
Jul 3, 2023
85aceb1
Merge branch 'master' into gn/verifiable-build
Jul 3, 2023
6b8a028
remove the message
Jul 3, 2023
00d82b4
calculate digest in separate func
Jul 3, 2023
14d1236
filter our status msgs in error
Jul 3, 2023
b2faaff
update comments
Jul 4, 2023
73b7166
Merge branch 'master' into gn/verifiable-build
Jul 4, 2023
4700472
resolve other merge conflicts
Jul 4, 2023
0c40092
display container name
Jul 10, 2023
14ab832
Merge branch 'master' into gn/verifiable-build
Jul 10, 2023
92a7867
properly trancate error output
Jul 10, 2023
517b4bf
Merge branch 'master' into gn/verifiable-build
Jul 12, 2023
b6b66e9
apply Andrews suggestions
Jul 12, 2023
590b5b8
proper build step logs
Jul 12, 2023
d515504
fix generation of ABI
Jul 12, 2023
44958e3
hash the cmd, not entrypoint for digest
Jul 12, 2023
ad1ddc6
fix args parsing and add docs
Jul 13, 2023
6e02958
clippy fix
Jul 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
333 changes: 117 additions & 216 deletions Cargo.lock

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions build-image/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
target/
74 changes: 74 additions & 0 deletions build-image/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
ARG VCS_REF=master
ARG REGISTRY_PATH=docker.io/paritytech
ARG BUILD_DATE

FROM ${REGISTRY_PATH}/base-ci-linux:latest

# metadata
LABEL io.parity.image.vendor="Parity Technologies" \
SkymanOne marked this conversation as resolved.
Show resolved Hide resolved
io.parity.image.title="${REGISTRY_PATH}/ink-build-image" \
io.parity.image.documentation="https://github.com/paritytech/scripts/blob/${VCS_REF}/\
dockerfiles/ink-ci-linux/README.md" \
io.parity.image.description="Inherits from docker.io/paritytech/base-ci-linux. \
rust nightly, clippy, rustfmt, miri, rust-src, rustc-dev, grcov, rust-covfix, \
llvm-tools-preview, cargo-contract, xargo, binaryen, parallel, codecov, ink, solang" \
io.parity.image.revision="${VCS_REF}" \
io.parity.image.documentation="https://github.com/paritytech/cargo-contract/blob/${VCS_REF}/\
build-image/README.md" \
io.parity.image.created="${BUILD_DATE}"

WORKDIR /builds

RUN set -eux; \
apt-get -y update && \

# `binaryen` is needed by `cargo-contract` for optimizing Wasm files.
# We fetch the latest release which contains a Linux binary.
curl -L $(curl --silent https://api.github.com/repos/WebAssembly/binaryen/releases \
| jq -r '.[0].assets | [.[] | .browser_download_url] | map(select(match("x86_64-linux\\.tar\\.gz$"))) | .[0]' \
) | tar -xz -C /usr/local/bin/ --wildcards --strip-components=2 'binaryen-*/bin/wasm-opt' && \

# Install LLVM for solang
curl -L https://github.com/hyperledger/solang-llvm/releases/download/llvm15-1/llvm15.0-linux-x86-64.tar.xz > llvm15.0-linux-x86-64.tar.xz && \
tar Jxf llvm15.0-linux-x86-64.tar.xz && \
export PATH=$(pwd)/llvm15.0/bin:$PATH && \

# The stable toolchain is used to build ink! itself through the use of the
# `RUSTC_BOOSTRAP=1` environment variable. We also need to install the
# `wasm32-unknown-unknown` target since that's the platform that ink! smart contracts
# run on.

# The 1.69 toolchain is temporarily required to build ink! contracts because of
# https://github.com/paritytech/cargo-contract/issues/1139 \
rustup toolchain install 1.69 && \
rustup target add wasm32-unknown-unknown --toolchain 1.69 && \
rustup component add rust-src clippy rustfmt --toolchain 1.69 && \
rustup default 1.69 && \

# `cargo-dylint` and `dylint-link` are dependencies needed to run `cargo-contract`.
cargo install cargo-dylint dylint-link && \

# Install the latest stable release of `cargo-contract`
cargo install cargo-contract && \

# Install the latest `solang`
cargo install solang && \

# Versions
rustup show && \
cargo --version && \
cargo-contract --version && \
wasm-opt --version && \

# Clean up and remove compilation artifacts that a cargo install creates (>250M).
rm -rf "${CARGO_HOME}/registry" "${CARGO_HOME}/git" /root/.cache/sccache && \

# apt clean up
apt-get remove -y gnupg && \
apt-get autoremove -y && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# A contract workdir which should be bind mounted to the container. See README for detailed documentation.
WORKDIR /contract

2 changes: 2 additions & 0 deletions build-image/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Verifiable build using Docker

4 changes: 4 additions & 0 deletions build-image/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
docker run -d \
--name flipper \
--mount type=bind,source="$(pwd)",target="/contract" \
parity/ver-build
3 changes: 3 additions & 0 deletions crates/build/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ wasm-opt = "0.112.0"
which = "4.4.0"
zip = { version = "0.6.6", default-features = false }
strum = { version = "0.24", features = ["derive"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
tokio-stream = "0.1"
bollard = "0.14"

contract-metadata = { version = "3.0.1", path = "../metadata" }

Expand Down
12 changes: 10 additions & 2 deletions crates/build/src/args.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ impl TryFrom<&VerbosityFlags> for Verbosity {
}

/// Denotes if output should be printed to stdout.
#[derive(Clone, Copy, Default, serde::Serialize, Eq, PartialEq)]
#[derive(Clone, Copy, Default, serde::Serialize, serde::Deserialize, Eq, PartialEq)]
pub enum Verbosity {
/// Use default output
#[default]
Expand Down Expand Up @@ -89,7 +89,15 @@ impl Network {

/// Describes which artifacts to generate
#[derive(
Copy, Clone, Default, Eq, PartialEq, Debug, clap::ValueEnum, serde::Serialize,
Copy,
Clone,
Default,
Eq,
PartialEq,
Debug,
clap::ValueEnum,
serde::Serialize,
serde::Deserialize,
)]
#[clap(name = "build-artifacts")]
pub enum BuildArtifacts {
Expand Down
Loading