Deprecate zigbuild and glibc in favor of static musl binaries for Lin…

…ux builds. (kaspanet#541) * CTNG Config file, Musl build instructions - Crosstools-ng configuration file for quick reproducable builds on musl. - Instructions for how to build RK on musl. * Test * Update ci.yaml * Test crosstools install * Cache ct-ng build * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Fix error in command * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Comments, naming * Update ci.yaml * Update ci.yaml * Update ci.yaml * Merge new musl build with old CI, Release builds - Merges the old CI script with the new musl build. - Update deploy.yaml to use updated musl toolchain to build musl target for linux. * Move to workspace * Delete musl-build.md * Lock to ctng version 1.26 * Checkout fix * Revert master change * Indentation * Revert "Indentation" This reverts commit 6a7e6c0. * Revert "Revert master change" This reverts commit 1a047e4. * Update ci.yaml * Force mimalloc * Compress into script * Fix typo * Update build.sh * Replace bloaded config file * Update build.sh * Update build.sh * Source script * Revert vendor * Update defconfig * Update defconfig * Update defconfig * Update build.sh * Update build.sh * Update build.sh * Update build.sh * Update defconfig * Delete defconfig * Create defconfig * Update build.sh * Deprecate config, use default preset * Update build.sh * Add preset hash logic in script * Move preset hash update Move preset hash update after openssl build * Use openssl crate * Update exports, cache config * Remove spaces in export command * Update names Should not trigger cache * Move source preset * CD before preset * Add comment Adds comment and should invalidate cache.
michaelsutton · Sep 5, 2024 · 06a874f · 06a874f
1 parent b04092e
commit 06a874f
Show file tree

Hide file tree

Showing 5 changed files with 144 additions and 25 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -353,9 +353,8 @@ jobs:
         with: 
           name: kaspa-wasm32-sdk-${{ env.SHORT_SHA }}.zip
           path: wasm/release/kaspa-wasm32-sdk-${{ env.SHORT_SHA }}.zip
-
   build-release:
-    name: Build Ubuntu Release
+    name: Build Linux Release
     runs-on: ubuntu-latest
     steps:
       - name: Checkout sources
@@ -369,7 +368,7 @@ jobs:
       - name: Install stable toolchain
         uses: dtolnay/rust-toolchain@stable
 
-      - name: Cache
+      - name: Cache Cargo Build Outputs
         uses: actions/cache@v4
         with:
           path: |
@@ -379,14 +378,23 @@ jobs:
             ~/.cargo/git/db/
             target/
           key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-cargo-
+        
+      - name: Cache Toolchain
+        uses: actions/cache@v4
+        with:
+          path: |
+            ~/x-tools
+          key: ${{ runner.os }}-musl-${{ hashFiles('**/musl-toolchain/preset.sh') }}
+          restore-keys: |
+            ${{ runner.os }}-musl-
 
-      - name: Install zig
-        if: runner.os == 'Linux'
-        uses: goto-bus-stop/setup-zig@v2 # needed for cargo-zigbuild
-
-      - name: Build on Linux
+      
+      - name: Build RK with musl toolchain
         if: runner.os == 'Linux'
-        # We're using musl to make the binaries statically linked and portable
         run: |
-          cargo install cargo-zigbuild
-          cargo --verbose zigbuild --bin kaspad --bin simpa --bin rothschild --release --target x86_64-unknown-linux-gnu.2.27 # Use an older glibc version
+          # Run build script for musl toolchain
+          source musl-toolchain/build.sh
+          # Build for musl
+          cargo --verbose build --bin kaspad --bin rothschild --bin kaspa-wallet --release --target x86_64-unknown-linux-musl
diff --git a/.github/workflows/deploy.yaml b/.github/workflows/deploy.yaml
@@ -29,7 +29,7 @@ jobs:
       - name: Install stable toolchain
         uses: dtolnay/rust-toolchain@stable
 
-      - name: Cache
+      - name: Cache Cargo Build Outputs
         uses: actions/cache@v3
         with:
           path: |
@@ -40,23 +40,30 @@ jobs:
             target/
           key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
 
-      - name: Install zig on linux
-        if: runner.os == 'Linux'
-        uses: goto-bus-stop/setup-zig@v2 # needed for cargo-zigbuild
+      - name: Cache Toolchain
+        uses: actions/cache@v4
+        with:
+          path: |
+            ~/x-tools
+          key: ${{ runner.os }}-musl-${{ hashFiles('**/musl-toolchain/preset.sh') }}
+          restore-keys: |
+            ${{ runner.os }}-musl-
 
       - name: Build on Linux
         if: runner.os == 'Linux'
         # We're using musl to make the binaries statically linked and portable
         run: |
-          cargo install cargo-zigbuild
-          cargo --verbose zigbuild --bin kaspad --bin simpa --bin rothschild --bin kaspa-wallet --release --target x86_64-unknown-linux-gnu.2.27 # Use an older glibc version
+          # Run build script for musl toolchain
+          source musl-toolchain/build.sh
+
+          # Build for musl
+          cargo --verbose build --bin kaspad --bin rothschild --bin kaspa-wallet --release --target x86_64-unknown-linux-musl
           mkdir bin || true
-          cp target/x86_64-unknown-linux-gnu/release/kaspad bin/
-          cp target/x86_64-unknown-linux-gnu/release/simpa bin/
-          cp target/x86_64-unknown-linux-gnu/release/rothschild bin/
-          cp target/x86_64-unknown-linux-gnu/release/kaspa-wallet bin/
-          archive="bin/rusty-kaspa-${{ github.event.release.tag_name }}-linux-gnu-amd64.zip"
-          asset_name="rusty-kaspa-${{ github.event.release.tag_name }}-linux-gnu-amd64.zip"
+          cp target/x86_64-unknown-linux-musl/release/kaspad bin/
+          cp target/x86_64-unknown-linux-musl/release/rothschild bin/
+          cp target/x86_64-unknown-linux-musl/release/kaspa-wallet bin/
+          archive="bin/rusty-kaspa-${{ github.event.release.tag_name }}-linux-musl-amd64.zip"
+          asset_name="rusty-kaspa-${{ github.event.release.tag_name }}-linux-musl-amd64.zip"
           zip -r "${archive}" ./bin/*
           echo "archive=${archive}" >> $GITHUB_ENV
           echo "asset_name=${asset_name}" >> $GITHUB_ENV

diff --git a/musl-toolchain/build.sh b/musl-toolchain/build.sh
@@ -0,0 +1,96 @@
+#!/bin/bash
+
+PRESET_HASH_FILE="$HOME/x-tools/preset_hash"
+
+# Calculate the hash of the preset file
+CURRENT_PRESET_HASH=$(sha256sum $GITHUB_WORKSPACE/musl-toolchain/preset.sh | awk '{print $1}')
+
+echo "Current preset hash: $CURRENT_PRESET_HASH"
+
+# Traverse to working directory
+cd $GITHUB_WORKSPACE/musl-toolchain
+
+# Set the preset
+source preset.sh
+
+# If the toolchain is not installed or the preset has changed or the preset hash file does not exist
+if [ ! -d "$HOME/x-tools" ] || [ ! -f "$PRESET_HASH_FILE" ] || [ "$(cat $PRESET_HASH_FILE)" != "$CURRENT_PRESET_HASH" ]; then
+  # Install dependencies
+  sudo apt-get update
+  sudo apt-get install -y autoconf automake libtool  libtool-bin unzip help2man python3.10-dev gperf bison flex texinfo gawk libncurses5-dev
+
+  # Clone crosstool-ng
+  git clone https://github.com/crosstool-ng/crosstool-ng
+
+  # Configure and build crosstool-ng
+  cd crosstool-ng
+  # Use version 1.26
+  git checkout crosstool-ng-1.26.0
+  ./bootstrap
+  ./configure --prefix=$HOME/ctng
+  make
+  make install
+  # Add crosstool-ng to PATH
+  export PATH=$HOME/ctng/bin:$PATH
+
+
+
+  # Load toolchainc configuration
+  ct-ng $CTNG_PRESET
+
+  # Build the toolchain
+  ct-ng build > build.log 2>&1
+
+  # Set status to the exit code of the build
+  status=$?
+
+  # We store the log in a file because it bloats the screen too much
+  # on GitHub Actions. We print it only if the build fails.
+  echo "Build result:"
+  if [ $status -eq 0 ]; then
+    echo "Build succeeded"
+    ls -la $HOME/x-tools
+    # Store the current hash of preset.sh after successful build
+    echo "$CURRENT_PRESET_HASH" > "$PRESET_HASH_FILE"    
+  else
+    echo "Build failed, here's the log:"
+    cat .config
+    cat build.log
+  fi
+fi
+
+# Update toolchain variables: C compiler, C++ compiler, linker, and archiver
+export CC=$HOME/x-tools/$CTNG_PRESET/bin/$CTNG_PRESET-gcc
+export CXX=$HOME/x-tools/$CTNG_PRESET/bin/$CTNG_PRESET-g++
+export LD=$HOME/x-tools/$CTNG_PRESET/bin/$CTNG_PRESET-ld
+export AR=$HOME/x-tools/$CTNG_PRESET/bin/$CTNG_PRESET-ar     
+
+# Exports for cc crate
+# https://docs.rs/cc/latest/cc/#external-configuration-via-environment-variables
+export RANLIB_x86_64_unknown_linux_musl=$HOME/x-tools/$CTNG_PRESET/bin/$CTNG_PRESET-ranlib     
+export CC_x86_64_unknown_linux_musl=$CC
+export CXX_x86_64_unknown_linux_musl=$CXX
+export AR_x86_64_unknown_linux_musl=$AR
+export LD_x86_64_unknown_linux_musl=$LD
+
+# Set environment variables for static linking
+export OPENSSL_STATIC=true
+export RUSTFLAGS="-C link-arg=-static"
+
+# We specify the compiler that will invoke linker
+export CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=$CC
+
+# Add target
+rustup target add x86_64-unknown-linux-musl
+
+# Install missing dependencies
+cargo fetch --target x86_64-unknown-linux-musl
+
+# Patch missing include in librocksdb-sys-0.16.0+8.10.0. Credit: @supertypo
+FILE_PATH=$(find $HOME/.cargo/registry/src/ -path "*/librocksdb-sys-0.16.0+8.10.0/*/offpeak_time_info.h")
+
+if [ -n "$FILE_PATH" ]; then
+  sed -i '1i #include <cstdint>' "$FILE_PATH"
+else
+  echo "No such file for sed modification."
+fi
diff --git a/musl-toolchain/preset.sh b/musl-toolchain/preset.sh
@@ -0,0 +1,4 @@
+#!/bin/bash
+# Sets the preset that will be used by crosstool-ng
+# Available presets can be fetched with: ct-ng list-samples
+export CTNG_PRESET="x86_64-multilib-linux-musl"
diff --git a/rpc/wrpc/server/Cargo.toml b/rpc/wrpc/server/Cargo.toml
@@ -35,8 +35,12 @@ workflow-log.workspace = true
 workflow-rpc.workspace = true
 workflow-serializer.workspace = true
 
-[target.x86_64-unknown-linux-gnu.dependencies]
-# Adding explicitely the openssl dependency here is needed for a successful build with zigbuild
+# Adding explicitely the openssl dependency here is needed for a successful build with zigbuild and musl
 # as used in the release deployment in GitHub CI
 # see: https://github.com/rust-cross/cargo-zigbuild/issues/127
+
+[target.x86_64-unknown-linux-gnu.dependencies]
+openssl = { version = "0.10", features = ["vendored"] }
+
+[target.x86_64-unknown-linux-musl.dependencies]
 openssl = { version = "0.10", features = ["vendored"] }