Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tilt: m1 flag #4175

Merged
Merged

Conversation

evan-gray
Copy link
Contributor

@evan-gray evan-gray commented Nov 29, 2024

This PR adds support for solana-test-validator within Docker / Tilt for M-series Macs. This is useful for downstream projects such as NTT which rely on a subset of the tilt setup for e2e testing. It may also benefit contributors who wish to run a minimal tilt setup on their Macs.

The key hold up was Rosetta 2 lacking AVX instruction support, which solana-test-validator relies upon. Everything would compile as amd64 but would not run. Further complicating efforts, there is not a pre-built binary for linux/aarch64. Unlike prior attempts to build the entire solana dependency under aarch64, this simply swaps out the binary used.

This same shortcoming may also prevent some other chains from running, such as Near. Further work could be done to provide aarch64 alternative dockerfiles for the relevant images which do not run properly as amd64 with Rosetta 2.

This PR also includes a change for the wormhole-deploy image, as the existing node image did not build.

Copy link
Contributor

@kcsongor kcsongor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat!

Copy link
Contributor

@johnsaigle johnsaigle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look OK but I couldn't get it to work while testing. Do you have any suggestions for verifying this locally?

Approach:

Run tilt up -- --solana --m1 to see if it builds

Steps:

  • Delete existing minikube via minikube delete
  • Starting minikube without kvm2 as I got an error that it's not supported on Mac: minikube start --cpus=8 --memory=8G --disk-size=50G
  • Run tilt up -- --solana --m1

Result

STEP 1/11 — Building Dockerfile: [solana-contract]
Building Dockerfile for platform linux/arm64:
  #syntax=docker/dockerfile:1.2@sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc
  FROM ghcr.io/wormhole-foundation/solana:1.10.31@sha256:d31e8db926a1d3fbaa9d9211d9979023692614b7b64912651aba0383e8c01bad AS solana
  
  # Add bridge contract sources
  WORKDIR /usr/src/bridge
  
  COPY bridge bridge
  COPY modules modules
  COPY migration migration
  COPY Cargo.toml Cargo.toml
  COPY Cargo.lock Cargo.lock
  COPY solitaire solitaire
  COPY external external
  
  ENV RUST_LOG="solana_runtime::system_instruction_processor=trace,solana_runtime::message_processor=trace,solana_bpf_loader=debug,solana_rbpf=debug"
  ENV RUST_BACKTRACE=1
  
  FROM solana AS builder
  
  RUN mkdir -p /opt/solana/deps
  
  ARG EMITTER_ADDRESS="11111111111111111111111111111115"
  ARG BRIDGE_ADDRESS
  RUN [ -n "${BRIDGE_ADDRESS}" ]
  
  # Build Wormhole Solana programs
  RUN --mount=type=cache,target=target,id=build \
      --mount=type=cache,target=/usr/local/cargo/registry,id=cargo_registry \
      cargo build-bpf --manifest-path "bridge/program/Cargo.toml" -- --locked && \
      cargo build-bpf --manifest-path "bridge/cpi_poster/Cargo.toml" -- --locked && \
      cargo build-bpf --manifest-path "modules/token_bridge/program/Cargo.toml" -- --locked && \
      cargo build-bpf --manifest-path "modules/nft_bridge/program/Cargo.toml" -- --locked && \
      cargo build-bpf --manifest-path "migration/Cargo.toml" -- --locked && \
      cp target/deploy/bridge.so /opt/solana/deps/bridge.so && \
      cp target/deploy/cpi_poster.so /opt/solana/deps/cpi_poster.so && \
      cp target/deploy/wormhole_migration.so /opt/solana/deps/wormhole_migration.so && \
      cp target/deploy/token_bridge.so /opt/solana/deps/token_bridge.so && \
      cp target/deploy/nft_bridge.so /opt/solana/deps/nft_bridge.so && \
      cp external/mpl_token_metadata.so /opt/solana/deps/mpl_token_metadata.so
  
  FROM scratch AS export-stage
  COPY --from=builder /opt/solana/deps /
  


     Building image
     resolve image config for docker-image://docker.io/docker/dockerfile:1.2@sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc [done: 547ms]
     docker-image://docker.io/docker/dockerfile:1.2@sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc 9.01MB / 9.01MB [done: 430ms]
     [solana 1/9] FROM ghcr.io/wormhole-foundation/solana:1.10.31@sha256:d31e8db926a1d3fbaa9d9211d9979023692614b7b64912651aba0383e8c01bad 1.25GB / 1.25GB [done: 58.276s]
     [background] read source files 1.43MB [done: 43ms]
     [solana 2/9] WORKDIR /usr/src/bridge [done: 117ms]
     [solana 3/9] COPY bridge bridge [done: 60ms]
     [solana 4/9] COPY modules modules [done: 26ms]
     [solana 5/9] COPY migration migration [done: 16ms]
     [solana 6/9] COPY Cargo.toml Cargo.toml [done: 33ms]
     [solana 7/9] COPY Cargo.lock Cargo.lock [done: 11ms]
     [solana 8/9] COPY solitaire solitaire [done: 18ms]
     [solana 9/9] COPY external external [done: 15ms]
     [builder 1/3] RUN mkdir -p /opt/solana/deps
       → exec /bin/sh: exec format error
     
     ERROR IN: [builder 1/3] RUN mkdir -p /opt/solana/deps

Build Failed: ImageBuild: process "/bin/sh -c mkdir -p /opt/solana/deps" did not complete successfully: exit code: 1

@evan-gray
Copy link
Contributor Author

@johnsaigle could you try with the latest Docker Desktop with Kubernetes enabled (per the Tilt install instructions for macOS).

As you mentioned, this command should be correct tilt up -- --solana --m1

Copy link
Contributor

@johnsaigle johnsaigle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working on latest Docker Desktop + Kubernetes

@evan-gray evan-gray merged commit db19128 into wormhole-foundation:main Dec 4, 2024
24 checks passed
@evan-gray evan-gray deleted the solana-m1-redux branch December 4, 2024 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants