Deprecate zigbuild and glibc in favor of static musl binaries for Linux builds. #541

starkbamse · 2024-09-03T15:51:29Z

What is this
This PR replaces current ways to build linux binaries (zigbuild) with a musl build using custom built OpenSSL and CTNG musl toolchains.

How
We utilize ctng to generate a musl toolchain using a configuration which includes libstdc++, a requirement by RocksDB. This allows us to compile OpenSSL and rusty-kaspa against musl.

Tests performed
Ran by two testers in #qa for two days.

Why do we need it
We need it because on certain machines (Ubuntu 22.04.1 LTS) when running TN10 we get a segmentation fault with the following backtrace snippet:

Core was generated by `./kaspad17800141 --testnet --utxoindex --disable-upnp --ram-scale=0.8 --appdir'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  rocksdb::CoreLocalArray<rocksdb::ConcurrentArena::Shard>::AccessElementAndIndex (this=0x48) at rocksdb/util/core_local.h:73
73  rocksdb/util/core_local.h: No such file or directory.
[Current thread is 1 (LWP 710665)]
warning: Missing auto-load script at offset 0 in section .debug_gdb_scripts
of file /home/starkbamse/coredump-old-rust/kaspad1780.
Use `info auto-load python-scripts [REGEXP]' to list them.
(gdb) bt
#0  rocksdb::CoreLocalArray<rocksdb::ConcurrentArena::Shard>::AccessElementAndIndex (this=0x48) at rocksdb/util/core_local.h:73
#1  rocksdb::ConcurrentArena::Repick (this=0x0) at rocksdb/memory/concurrent_arena.cc:38
#2  0x00005c0fbe499273 in rocksdb::ConcurrentArena::AllocateImpl<rocksdb::ConcurrentArena::AllocateAligned(unsigned long, unsigned long, rocksdb::Logger*)::{lambda()#1}>(unsigned long, bool, rocksdb::ConcurrentArena::AllocateAligned(unsigned long, unsigned long, rocksdb::Logger*)::{lambda()#1} const&) (this=0x471e51638a0, bytes=56, force_arena=<optimized out>, func=...)

Cache updates
You can trigger a cache update by updating the preset.sh. Both Actions, and the build.sh script keep track of the latest hash and rebuild in the case of a mismatch.

Considerations:
Performance, could be worse if areas where performance is crucial utilize C memory allocators. It is planned to maybe replace the allocator with mimalloc to solve this problem.

- Crosstools-ng configuration file for quick reproducable builds on musl. - Instructions for how to build RK on musl.

…ild-hello-musl

Move preset hash update after openssl build

musl-toolchain/build.sh

Should not trigger cache

Adds comment and should invalidate cache.

rpc/wrpc/server/Cargo.toml

elichai

Nice! THis looks great now :)

commit afbcf9e Author: starkbamse <139136798+starkbamse@users.noreply.github.com> Date: Fri Sep 6 05:40:53 2024 +0200 Change directory back to repo root & Fix Rust v1.81 lints (kaspanet#545) * Change directory back to repodir Change directory back to repodir after building toolchain * Clippy * Update crypto/txscript/src/caches.rs Co-authored-by: Maxim <59533214+biryukovmaxim@users.noreply.github.com> * Update crypto/txscript/src/caches.rs * rename `is_none_or` -> `is_none_or_ex` to avoid conflict with future std * remove `use std::mem::size_of` wherever possible (added to std prelude recently) --------- Co-authored-by: Maxim <59533214+biryukovmaxim@users.noreply.github.com> Co-authored-by: Michael Sutton <msutton@cs.huji.ac.il> commit 06a874f Author: starkbamse <139136798+starkbamse@users.noreply.github.com> Date: Thu Sep 5 18:51:04 2024 +0200 Deprecate zigbuild and glibc in favor of static musl binaries for Linux builds. (kaspanet#541) * CTNG Config file, Musl build instructions - Crosstools-ng configuration file for quick reproducable builds on musl. - Instructions for how to build RK on musl. * Test * Update ci.yaml * Test crosstools install * Cache ct-ng build * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Fix error in command * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Update ci.yaml * Comments, naming * Update ci.yaml * Update ci.yaml * Update ci.yaml * Merge new musl build with old CI, Release builds - Merges the old CI script with the new musl build. - Update deploy.yaml to use updated musl toolchain to build musl target for linux. * Move to workspace * Delete musl-build.md * Lock to ctng version 1.26 * Checkout fix * Revert master change * Indentation * Revert "Indentation" This reverts commit 6a7e6c0. * Revert "Revert master change" This reverts commit 1a047e4. * Update ci.yaml * Force mimalloc * Compress into script * Fix typo * Update build.sh * Replace bloaded config file * Update build.sh * Update build.sh * Source script * Revert vendor * Update defconfig * Update defconfig * Update defconfig * Update build.sh * Update build.sh * Update build.sh * Update build.sh * Update defconfig * Delete defconfig * Create defconfig * Update build.sh * Deprecate config, use default preset * Update build.sh * Add preset hash logic in script * Move preset hash update Move preset hash update after openssl build * Use openssl crate * Update exports, cache config * Remove spaces in export command * Update names Should not trigger cache * Move source preset * CD before preset * Add comment Adds comment and should invalidate cache. commit b04092e Author: aspect <anton.yemelyanov@gmail.com> Date: Thu Sep 5 18:00:24 2024 +0300 add proxy limit field to sysinfo (kaspanet#544) commit f866dfa Author: Michael Sutton <msutton@cs.huji.ac.il> Date: Thu Sep 5 14:04:16 2024 +0300 Various miscellaneous changes towards 0.15.1 RC2 (kaspanet#543) * infrequent logs should be debug * cleanup some todos * when a network starts, genesis has a body, so there's no need for a special exception * remove unneeded method and add an error just in case it is added in the future * count and log chain disqualified blocks * count and log mempool evictions * bump version to 0.14.5

starkbamse added 14 commits September 3, 2024 01:26

CTNG Config file, Musl build instructions

7eb30c0

- Crosstools-ng configuration file for quick reproducable builds on musl. - Instructions for how to build RK on musl.

Test

81a9155

Update ci.yaml

8db655c

Test crosstools install

5fcedaa

Cache ct-ng build

d01aaae

Update ci.yaml

c7002f6

Update ci.yaml

969ef80

Update ci.yaml

494e692

Update ci.yaml

6dfe52e

Update ci.yaml

bf1700a

Update ci.yaml

5beb588

Update ci.yaml

ae0e535

Update ci.yaml

cc708fe

Update ci.yaml

48cc7fc

starkbamse changed the title ~~Goodbye zigbuild hello musl~~ Deprecate zigbuild and glibc in favor of static musl binaries for Linux builds. Sep 4, 2024

starkbamse marked this pull request as draft September 4, 2024 09:10

starkbamse added 14 commits September 4, 2024 11:26

Update ci.yaml

c003e26

Update ci.yaml

f0467cc

Update ci.yaml

b7d786e

Fix error in command

c3f9480

Update ci.yaml

7af4cc8

Update ci.yaml

c8fb762

Update ci.yaml

ca28149

Update ci.yaml

e9ad9ef

Update ci.yaml

dd76647

Update ci.yaml

1689292

Update ci.yaml

3531a90

Update ci.yaml

52fa419

Update ci.yaml

b928b9a

Update ci.yaml

dee8e95

starkbamse added 14 commits September 5, 2024 13:11

Update build.sh

425ce81

Update build.sh

e67d728

Update build.sh

a0f21c3

Update build.sh

b5057c6

Update defconfig

75c5ad2

Delete defconfig

705d1d8

Create defconfig

adf597f

Update build.sh

90541bc

Deprecate config, use default preset

24fdc12

Merge branch 'dev' into goodbye-zigbuild-hello-musl

60a8a4f

Merge branch 'goodbye-zigbuild-hello-musl-workdir' into goodbye-zigbu…

4f8ebb2

…ild-hello-musl

Update build.sh

8a0ea58

Add preset hash logic in script

7230ec0

Move preset hash update

74654b2

Move preset hash update after openssl build

starkbamse assigned starkbamse and unassigned starkbamse Sep 5, 2024

elichai reviewed Sep 5, 2024

View reviewed changes

musl-toolchain/build.sh Outdated Show resolved Hide resolved

musl-toolchain/build.sh Outdated Show resolved Hide resolved

musl-toolchain/build.sh Outdated Show resolved Hide resolved

starkbamse added 8 commits September 5, 2024 15:29

Use openssl crate

6d6faad

Update exports, cache config

80458d8

Remove spaces in export command

6337b1b

Update names

e67a519

Should not trigger cache

Merge branch 'dev' into goodbye-zigbuild-hello-musl

a390f34

Move source preset

50be96b

CD before preset

be6a369

Add comment

6bfff24

Adds comment and should invalidate cache.

KaffinPX reviewed Sep 5, 2024

View reviewed changes

rpc/wrpc/server/Cargo.toml Show resolved Hide resolved

michaelsutton approved these changes Sep 5, 2024

View reviewed changes

elichai approved these changes Sep 5, 2024

View reviewed changes

michaelsutton merged commit 06a874f into kaspanet:dev Sep 5, 2024
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deprecate zigbuild and glibc in favor of static musl binaries for Linux builds. #541

Deprecate zigbuild and glibc in favor of static musl binaries for Linux builds. #541

starkbamse commented Sep 3, 2024 •

edited

Loading

elichai left a comment

Deprecate zigbuild and glibc in favor of static musl binaries for Linux builds. #541

Deprecate zigbuild and glibc in favor of static musl binaries for Linux builds. #541

Conversation

starkbamse commented Sep 3, 2024 • edited Loading

elichai left a comment

Choose a reason for hiding this comment

starkbamse commented Sep 3, 2024 •

edited

Loading