Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make portable the default in Lighthouse book and Makefile, update workflows #6247

Merged
merged 16 commits into from
Aug 19, 2024
Merged
3 changes: 0 additions & 3 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -124,9 +124,6 @@ jobs:
push: true
tags: |
${{ github.repository_owner}}/${{ matrix.binary }}:${{ env.VERSION }}-${{ env.SHORT_ARCH }}${{ env.VERSION_SUFFIX }}
${{ github.repository_owner}}/${{ matrix.binary }}:${{ env.VERSION }}-${{ env.SHORT_ARCH }}${{ env.VERSION_SUFFIX }}-dev
${{ github.repository_owner}}/${{ matrix.binary }}:${{ env.VERSION }}-${{ env.SHORT_ARCH }}${{ env.VERSION_SUFFIX }}-modern
${{ github.repository_owner}}/${{ matrix.binary }}:${{ env.VERSION }}-${{ env.SHORT_ARCH }}${{ env.VERSION_SUFFIX }}-modern-dev

- name: Build and push (lcli)
if: startsWith(matrix.binary, 'lcli')
Expand Down
4 changes: 0 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -253,13 +253,9 @@ jobs:
| System | Architecture | Binary | PGP Signature |
|:---:|:---:|:---:|:---|
| <img src="https://simpleicons.org/icons/apple.svg" style="width: 32px;"/> | x86_64 | [lighthouse-${{ env.VERSION }}-x86_64-apple-darwin.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-apple-darwin.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-apple-darwin.tar.gz.asc) |
| <img src="https://simpleicons.org/icons/apple.svg" style="width: 32px;"/> | x86_64 | [lighthouse-${{ env.VERSION }}-x86_64-apple-darwin-portable.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-apple-darwin-portable.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-apple-darwin-portable.tar.gz.asc) |
| <img src="https://simpleicons.org/icons/linux.svg" style="width: 32px;"/> | x86_64 | [lighthouse-${{ env.VERSION }}-x86_64-unknown-linux-gnu.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-unknown-linux-gnu.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-unknown-linux-gnu.tar.gz.asc) |
| <img src="https://simpleicons.org/icons/linux.svg" style="width: 32px;"/> | x86_64 | [lighthouse-${{ env.VERSION }}-x86_64-unknown-linux-gnu-portable.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-unknown-linux-gnu-portable.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-unknown-linux-gnu-portable.tar.gz.asc) |
| <img src="https://simpleicons.org/icons/raspberrypi.svg" style="width: 32px;"/> | aarch64 | [lighthouse-${{ env.VERSION }}-aarch64-unknown-linux-gnu.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-aarch64-unknown-linux-gnu.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-aarch64-unknown-linux-gnu.tar.gz.asc) |
| <img src="https://simpleicons.org/icons/raspberrypi.svg" style="width: 32px;"/> | aarch64 | [lighthouse-${{ env.VERSION }}-aarch64-unknown-linux-gnu-portable.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-aarch64-unknown-linux-gnu-portable.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-aarch64-unknown-linux-gnu-portable.tar.gz.asc) |
| <img src="https://upload.wikimedia.org/wikipedia/commons/c/c4/Windows_logo_-_2021_%28Black%29.svg" style="width: 32px;"/> | x86_64 | [lighthouse-${{ env.VERSION }}-x86_64-windows.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-windows.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-windows.tar.gz.asc) |
| <img src="https://upload.wikimedia.org/wikipedia/commons/c/c4/Windows_logo_-_2021_%28Black%29.svg" style="width: 32px;"/> | x86_64 | [lighthouse-${{ env.VERSION }}-x86_64-windows-portable.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-windows-portable.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/lighthouse-${{ env.VERSION }}-x86_64-windows-portable.tar.gz.asc) |
| | | | |
| **System** | **Option** | - | **Resource** |
| <img src="https://simpleicons.org/icons/docker.svg" style="width: 32px;"/> | Docker | [${{ env.VERSION }}](https://hub.docker.com/r/${{ env.IMAGE_NAME }}/tags?page=1&ordering=last_updated&name=${{ env.VERSION }}) | [${{ env.IMAGE_NAME }}](https://hub.docker.com/r/${{ env.IMAGE_NAME }}) |
Expand Down
12 changes: 1 addition & 11 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -62,16 +62,10 @@ install-lcli:
#
# The resulting binaries will be created in the `target/` directory.
#
# The *-portable options compile the blst library *without* the use of some
# optimized CPU functions that may not be available on some systems. This
# results in a more portable binary with ~20% slower BLS verification.
# The *-portable options is the default feature.
build-x86_64:
cross build --bin lighthouse --target x86_64-unknown-linux-gnu --features "modern,$(CROSS_FEATURES)" --profile "$(CROSS_PROFILE)" --locked
build-x86_64-portable:
cross build --bin lighthouse --target x86_64-unknown-linux-gnu --features "portable,$(CROSS_FEATURES)" --profile "$(CROSS_PROFILE)" --locked
build-aarch64:
cross build --bin lighthouse --target aarch64-unknown-linux-gnu --features "$(CROSS_FEATURES)" --profile "$(CROSS_PROFILE)" --locked
build-aarch64-portable:
cross build --bin lighthouse --target aarch64-unknown-linux-gnu --features "portable,$(CROSS_FEATURES)" --profile "$(CROSS_PROFILE)" --locked

build-lcli-x86_64:
Expand All @@ -96,12 +90,8 @@ build-release-tarballs:
[ -d $(BIN_DIR) ] || mkdir -p $(BIN_DIR)
$(MAKE) build-x86_64
$(call tarball_release_binary,$(BUILD_PATH_X86_64),$(X86_64_TAG),"")
$(MAKE) build-x86_64-portable
$(call tarball_release_binary,$(BUILD_PATH_X86_64),$(X86_64_TAG),"-portable")
chong-he marked this conversation as resolved.
Show resolved Hide resolved
$(MAKE) build-aarch64
$(call tarball_release_binary,$(BUILD_PATH_AARCH64),$(AARCH64_TAG),"")
$(MAKE) build-aarch64-portable
$(call tarball_release_binary,$(BUILD_PATH_AARCH64),$(AARCH64_TAG),"-portable")

# Runs the full workspace tests in **release**, without downloading any additional
# test vectors.
Expand Down
10 changes: 1 addition & 9 deletions book/src/cross-compiling.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,10 @@ project.

### Targets

The `Makefile` in the project contains four targets for cross-compiling:
The `Makefile` in the project contains two targets for cross-compiling:

- `build-x86_64`: builds an optimized version for x86_64 processors (suitable for most users).
- `build-x86_64-portable`: builds a version for x86_64 processors which avoids using some modern CPU
instructions that are incompatible with older CPUs.
- `build-aarch64`: builds an optimized version for 64-bit ARM processors (suitable for Raspberry Pi 4).
- `build-aarch64-portable`: builds a version for 64-bit ARM processors which avoids using some
modern CPU instructions. In practice, very few ARM processors lack the instructions necessary to
run the faster non-portable build.

For more information about optimized vs portable builds see
[Portability](./installation-binaries.md#portability).

### Example

Expand Down
27 changes: 6 additions & 21 deletions book/src/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,28 +35,23 @@ Lighthouse vx.x.xx-xxxxxxxxx
BLS Library: xxxx-xxxxxxx
```

> Pro tip: try the `latest-modern` image for a 20-30% speed-up! See [Available Docker
> Images](#available-docker-images) below.

### Available Docker Images

There are several images available on Docker Hub.

Most users should use the `latest-modern` tag, which corresponds to the latest stable release of
Lighthouse with optimizations enabled. If you are running on older hardware then the default
`latest` image bundles a _portable_ version of Lighthouse which is slower but with better hardware
compatibility (see [Portability](./installation-binaries.md#portability)).
Most users should use the `latest` tag, which corresponds to the latest stable release of
Lighthouse with optimizations enabled.

To install a specific tag (in this case `latest-modern`), add the tag name to your `docker` commands:
To install a specific tag (in this case `latest`), add the tag name to your `docker` commands:

```bash
docker pull sigp/lighthouse:latest-modern
docker pull sigp/lighthouse:latest
```

Image tags follow this format:

```text
${version}${arch}${stability}${modernity}${features}
${version}${arch}${stability}
```

The `version` is:
Expand All @@ -75,19 +70,9 @@ The `stability` is:
* `-unstable` for the `unstable` branch
* empty for a tagged release or the `stable` branch

The `modernity` is:

* `-modern` for optimized builds
* empty for a `portable` unoptimized build

The `features` is:

* `-dev` for a development build with `minimal` preset enabled (`spec-minimal` feature).
* empty for a standard build with no custom feature enabled.

Examples:

* `latest-unstable-modern`: most recent `unstable` build for all modern CPUs (x86_64 or ARM)
* `latest-unstable`: most recent `unstable` build
* `latest-amd64`: most recent Lighthouse release for older x86_64 CPUs
* `latest-amd64-unstable`: most recent `unstable` build for older x86_64 CPUs

Expand Down
34 changes: 0 additions & 34 deletions book/src/installation-binaries.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,6 @@ Binaries are supplied for four platforms:
- `x86_64-apple-darwin`: macOS with Intel chips
- `x86_64-windows`: Windows with 64-bit processors

Additionally there is also a `-portable` suffix which indicates if the `portable` feature is used:

- Without `portable`: uses modern CPU instructions to provide the fastest signature verification times (may cause `Illegal instruction` error on older CPUs)
- With `portable`: approx. 20% slower, but should work on all modern 64-bit processors.

For details, see [Portability](#portability).

## Usage

Each binary is contained in a `.tar.gz` archive. For this example, lets assume the user needs
Expand All @@ -41,30 +34,3 @@ a `x86_64` binary.
1. (Optional) Move the `lighthouse` binary to a location in your `PATH`, so the `lighthouse` command can be called from anywhere. For example, to copy `lighthouse` from the current directory to `usr/bin`, run `sudo cp lighthouse /usr/bin`.

> Windows users will need to execute the commands in Step 2 from PowerShell.

## Portability

Portable builds of Lighthouse are designed to run on the widest range of hardware possible, but
sacrifice the ability to make use of modern CPU instructions.

If you have a modern CPU then you should try running a non-portable build to get a 20-30% speed up.

- For **x86_64**, any CPU supporting the [ADX](https://en.wikipedia.org/wiki/Intel_ADX) instruction set
extension is compatible with the optimized build. This includes Intel Broadwell (2014)
and newer, and AMD Ryzen (2017) and newer.
- For **ARMv8**, most CPUs are compatible with the optimized build, including the Cortex-A72 used by
the Raspberry Pi 4.

## Troubleshooting

If you get a SIGILL (exit code 132), then your CPU is incompatible with the optimized build
of Lighthouse and you should switch to the `-portable` build. In this case, you will see a
warning like this on start-up:

```
WARN CPU seems incompatible with optimized Lighthouse build, advice: If you get a SIGILL, please try Lighthouse portable build
```

On some VPS providers, the virtualization can make it appear as if CPU features are not available,
even when they are. In this case you might see the warning above, but so long as the client
continues to function, it's nothing to worry about.
3 changes: 1 addition & 2 deletions book/src/installation-source.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,8 +160,7 @@ FEATURES=gnosis,slasher-lmdb make
Commonly used features include:

- `gnosis`: support for the Gnosis Beacon Chain.
- `portable`: support for legacy hardware.
- `modern`: support for exclusively modern hardware.
- `portable`: the default feature as Lighthouse now uses runtime detection of hardware CPU features.
- `slasher-lmdb`: support for the LMDB slasher backend. Enabled by default.
- `slasher-mdbx`: support for the MDBX slasher backend.
- `jemalloc`: use [`jemalloc`][jemalloc] to allocate memory. Enabled by default on Linux and macOS.
Expand Down
Loading