Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build(Dockerfile): add Dockerfile and zombienet for parachain #124

Merged
merged 15 commits into from
Jul 18, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,22 @@ release: lint
testnet: release
zombienet -p native spawn zombienet/local-testnet.toml

build-parachain-docker:
docker build \
--build-arg VCS_REF="$(git rev-parse HEAD)" \
--build-arg BUILD_DATE="$(date -u +'%Y-%m-%dT%H:%M:%SZ')" \
-t polkadotstorage.azurecr.io/parachain-node:0.1.0 \
--file ./docker/dockerfiles/parachain/Dockerfile \
.

load-to-minikube:
# https://github.com/paritytech/zombienet/pull/1830
# unless this is merged and we pull it in, launching it in local zombienet (without publicly publishing the docker image is impossible)
minikube image load polkadotstorage.azurecr.io/parachain-node:0.1.0

kube-testnet:
zombienet -p kubernetes spawn zombienet/local-kube-testnet.toml

# Must be in sync with .vscode/settings.json and have extension Coverage Gutters to display it in VS Code.
market-coverage:
mkdir -p coverage
Expand Down
61 changes: 61 additions & 0 deletions PARACHAIN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Polka Storage Node - Parachain

Supported Kubernetes Platforms:
- Linux x86_64

Others were not tested and `polkadot` does not have an image for Linux ARM.

## Build the Docker Image

```bash
just build-parachain-docker
```

The command builds a Docker image `polkadotstorage.azurecr.io/parachain-node:0.1.0` from a Dockerfile located at `./docker/dockerfiles/parachain/Dockerfile`.

## Authenticating and Authorizing to Azure Container Registry

```bash
az login --use-device-code --tenant dareneiger.onmicrosoft.com
az acr login --name polkadotstorage.azurecr.io
th7nder marked this conversation as resolved.
Show resolved Hide resolved
```

> [!NOTE]
> Azure Container Registry token expires after 3 hours.

## Pulling the image from registry

Use this option if you don't want to build locally.

```bash
docker pull polkadotstorage.azurecr.io/parachain-node:0.1.0
```

## Running the Parachain

Prerequisites:
- Kubernetes Cluster access - configured [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl), e.g. [minikube](https://minikube.sigs.k8s.io/docs/start/),
- If using `minikube`, a started minikube cluster (`minikube start`).

The configuration is stored in `./zombienet/local-kube-testnet.toml`.
ZombieNet [does not support private Docker registries](https://github.com/paritytech/zombienet/issues/1829) we need to do some trickery to test it out in Kubernetes.
th7nder marked this conversation as resolved.
Show resolved Hide resolved

It requires:
1. Loading the image into the local minikube cluster
```bash
just load-to-minikube
```
2. Building patched ZombieNet
- requires NodeJS (LTS v20): preferably via [nvm](https://nodejs.org/en/download/package-manager)
- pulling a branch of [ZombieNet](https://github.com/paritytech/zombienet/pull/1830) and building it locally
```bash
git clone -b th7nder/fix/generating-containers git@github.com:th7nder/zombienet.git patched-zombienet
cd patched-zombienet/javascript
npm i && npm run build
npm run package
```
- NOTE: warnings like `> Warning Failed to make bytecode node18-arm64 for file` are normal and don't break the build.
3. Finally running patched ZombieNet (inside of `polka-storage` workspace)
```bash
patched-zombienet/javascript/bins/zombienet-linux-x64 -p kubernetes spawn zombienet/local-kube-testnet.toml
```
66 changes: 66 additions & 0 deletions docker/dockerfiles/parachain/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
################
##### Chef
FROM rust:1.77 AS chef
RUN cargo install cargo-chef
WORKDIR /app

################
##### Planer
FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

################
##### Builder
FROM chef AS builder

RUN apt update && apt upgrade -y
RUN apt install -y protobuf-compiler
RUN apt install -y clang

# Copy required files
COPY --from=planner /app/recipe.json recipe.json
COPY --from=planner /app/rust-toolchain.toml rust-toolchain.toml
# Build dependencies - this is the caching Docker layer!
RUN cargo chef cook --release --recipe-path recipe.json
# Build application
COPY . .
RUN cargo build --release -p polka-storage-node

FROM debian:bookworm-slim AS runtime

ARG VCS_REF
ARG BUILD_DATE

# show backtraces
ENV RUST_BACKTRACE=1

USER root

COPY --from=builder /app/target/release/polka-storage-node /usr/local/bin
RUN chmod -R a+rx "/usr/local/bin"

RUN useradd -m -u 1000 -U -s /bin/sh -d /eiger eiger
USER eiger
# check if executable works in this container
RUN /usr/local/bin/polka-storage-node --version

# 30333 for parachain p2p
# 30334 for relaychain p2p
# 9933 for RPC port
# 9944 for Websocket
# 9615 for Prometheus (metrics)

EXPOSE 30333 30334 9933 9944 9615
# mount-point for saving state of the parachain (not required for ZombieNet)
VOLUME ["/eiger"]
LABEL co.eiger.image.authors="releases@eiger.co" \
co.eiger.image.vendor="Eiger" \
co.eiger.image.title="Polka Storage Parachain" \
co.eiger.image.description="Parachain Node binary for Polka Storage, without specs. For local ZombieNet usage only." \
co.eiger.image.source="https://github.com/eigerco/polka-storage/blob/${VCS_REF}/docker/dockerfiles/parachain/Dockerfile" \
co.eiger.image.revision="${VCS_REF}" \
co.eiger.image.created="${BUILD_DATE}" \
co.eiger.image.documentation="https://github.com/eigerco/polka-storage"

ENTRYPOINT ["/usr/local/bin/polka-storage-node"]
2 changes: 2 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
};
rustToolchain = pkgs.pkgsBuildHost.rust-bin.fromRustupToolchainFile ./rust-toolchain.toml;
buildInputs = with pkgs; [
# Building Docker images and publishing to Azure Container Registry
azure-cli
cargo-llvm-cov
clang
pkg-config
Expand Down
31 changes: 31 additions & 0 deletions zombienet/local-kube-testnet.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
[settings]
image_pull_policy = "IfNotPresent"

[relaychain]
chain = "rococo-local"
default_args = ["--detailed-log-output", "-lparachain=debug,xcm=trace,runtime=trace"]
default_image = "docker.io/parity/polkadot:latest"

[[relaychain.nodes]]
name = "alice"
validator = true

[[relaychain.nodes]]
name = "bob"
validator = true

[[parachains]]
cumulus_based = true

# We need to use a Parachain of an existing System Chain (https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/runtime/rococo/src/xcm_config.rs).
# The reason: being able to get native DOTs from Relay Chain to Parachain via XCM Teleport.
# We'll have a proper Parachain ID in the *future*, but for now, let's stick to 1000 (which is AssetHub and trusted).
id = 1000

# run charlie as parachain collator
[[parachains.collators]]
args = ["--detailed-log-output", "-lparachain=debug,xcm=trace,runtime=trace"]
command = "polka-storage-node"
image = "polkadotstorage.azurecr.io/parachain-node:0.1.0"
name = "charlie"
validator = true