Illegal instruction (core dumped) in linux/arm/v6

[origamiofficial]

Describe the bug
When using armel arch, on say a Rpi 1B+, it will fail to start with:

/unbound.sh: line 420: 274 Illegal instruction (core dumped) /opt/unbound/sbin/unbound-anchor -a /opt/unbound/etc/unbound/var/root.key
Illegal instruction (core dumped)

How unbound was build:

ENV NAME=unbound \
    UNBOUND_VERSION=latest \
    UNBOUND_DOWNLOAD_URL=https://nlnetlabs.nl/downloads/unbound/unbound-latest.tar.gz

WORKDIR /tmp/src

COPY --from=openssl /opt/openssl /opt/openssl

RUN build_deps="curl gcc libc-dev libevent-dev libexpat1-dev libnghttp2-dev make flex bison" && \
    set -x && \
    DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y --no-install-recommends \
      $build_deps \
      bsdmainutils \
      ca-certificates \
      ldnsutils \
      libevent-2.1-7 \
      libexpat1 \
      libprotobuf-c-dev \
      protobuf-c-compiler && \
    curl -sSL $UNBOUND_DOWNLOAD_URL -o unbound.tar.gz && \
    tar xzf unbound.tar.gz && \
    rm -f unbound.tar.gz && \
    mv unbound-* unbound && \
    cd unbound && \
    groupadd _unbound && \
    useradd -g _unbound -s /dev/null -d /etc _unbound && \
    ./configure \
        --disable-dependency-tracking \
        --prefix=/opt/unbound \
        --with-pthreads \
        --with-username=_unbound \
        --with-ssl=/opt/openssl \
        --with-libevent \
        --with-libnghttp2 \
        --enable-dnstap \
        --enable-tfo-server \
        --enable-tfo-client \
        --enable-event-api \
        --enable-subnet && \
    make install && \
    mv /opt/unbound/etc/unbound/unbound.conf /opt/unbound/etc/unbound/unbound.conf.example && \
    apt-get purge -y --auto-remove \
      $build_deps && \
    rm -rf \
        /opt/unbound/share/man \
        /tmp/* \
        /var/tmp/* \
        /var/cache/apt/* \
        /var/lib/apt/lists/*

Full Dockerfile available here.

Note

It runs fine on all other architectures!

To reproduce
Steps to reproduce the behavior:

1. run command /opt/unbound/sbin/unbound-anchor -a /opt/unbound/etc/unbound/var/root.key

Expected behavior
It should run fine as it does on all other architectures except armel/ARMv6

System:

• Unbound version: 1.21.0
• OS: Debian GNU/Linux 11 (bullseye)
• unbound -V output:

Version 1.21.0

Configure line: --disable-dependency-tracking --prefix=/opt/unbound --with-pthreads --with-username=_unbound --with-ssl=/opt/openssl --with-libevent --with-libnghttp2 --enable-dnstap --enable-tfo-server --enable-tfo-client --enable-event-api --enable-subnet
Linked libs: libevent 2.1.12-stable (it uses epoll), OpenSSL 3.4.0-dev
Linked modules: dns64 subnetcache respip validator iterator
TCP Fastopen feature available

Additional information
Unbound was started using a unbound.sh script available here. Updated the unbound.sh file to list the opt/unbound/sbin dir and just run the unbound-anchor command without args:

...
drwxr-xr-x 2 root root     4096 Aug 15 15:28 .
drwxrwxr-x 1 root root     4096 Aug 15 09:41 ..
-rwxr-xr-x 1 root root 10668240 Aug 15 15:27 unbound
-rwxr-xr-x 1 root root  4688360 Aug 15 15:27 unbound-anchor
-rwxr-xr-x 1 root root  9197304 Aug 15 15:27 unbound-checkconf
-rwxr-xr-x 1 root root  6178844 Aug 15 15:27 unbound-control
-rwxr-xr-x 1 root root     6143 Aug 15 15:28 unbound-control-setup
-rwxr-xr-x 1 root root  4748440 Aug 15 15:27 unbound-host
  [i] Neutrino emissions detected...
/unbound.sh: line 421:   283 Illegal instruction     (core dumped) /opt/unbound/sbin/unbound-anchor
/unbound.sh: line 427:   296 Illegal instruction     (core dumped) /opt/unbound/sbin/unbound-anchor -a /opt/unbound/etc/unbound/var/root.key
...

Looks to me like building unbound in this way will successfully build, doesn't work on the linux/arm/v6

[wcawijngaards]

Other users report issues with 32bit vs 64bit versions for rpi compilation. Perhaps this is a cause of the issue, what was the unbound compiled for, eg. it prints that at the configure stage, and what is the running architecture. It may also be possible to run the other binaries, say with -h, to give a brief attempt to run it.

[cspoonerprice]

User that initially reported to origamioffical here. It was compiled on an x86 machine, but using the v6 platform flag.

docker buildx build --platform linux/arm/v6 -t unbound-pihole

The running arch on the rpi 1 B+:

$ uname -m
armv6l

OS is Raspbian GNU/Linux 12 (bookworm)

Updated the configure to capture output:

    ./configure \
        --disable-dependency-tracking \
        --prefix=/opt/unbound \
        --with-pthreads \
        --with-username=_unbound \
        --with-ssl=/opt/openssl \
        --with-libevent \
        --with-libnghttp2 \
        --enable-dnstap \
        --enable-tfo-server \
        --enable-tfo-client \
        --enable-event-api \
        --enable-subnet > /root/configure-output.log 2>&1 && \
    make install && \

I checked the configure-output.log, but no mention of architecture was made, it's quite a long file, and just restated the above ./configure from what I could tell.

running the unbound command itself also results in the core dump:

/unbound.sh: line 422:   283 Illegal instruction     (core dumped) unbound -v

Possibly related, I did find that the build of openssl (referenced above in the full docker file) only mentioned armv4 and armv7:

        + ./config --prefix=/opt/openssl --openssldir=/opt/openssl no-weak-ssl-ciphers no-ssl3 no-shared -DOPENSSL_NO_HEARTBEATS -fstack-protector-strong
        Configuring OpenSSL version 3.4.0-dev for target linux-armv4

Unsure if it's red herring, so I'm rebuilding the openssl with --target=armv6-linux-gnueabihf, but my system is quite slow and takes ~6h.

[cspoonerprice]

Was unable to get a successful build openssl with the armv6 flag. However, I determined that it wasn't the issue. After a lot of testing, I found I can build unbound with the --platform linux/arm/v6 flag, but only if the built and provided version of openssl is a bit older.

Using openssl versions 3.x (e.g. 3.4.0-dev) came up with the error "configure: error: OpenSSL found in /opt/openssl, but version 0.9.7 or higher is required". My testing wasn't exhaustive for the 3.x versions, as each build takes ~6h to run on my system.

If I changed to 1.1.1-stable, unbound was able to correctly detect the correct version and build/run.

Final working steps were to build with:

$ sudo docker buildx build --platform linux/arm/v6 -t unbound-pihole

using the dockerfile @origamiofficial linked, but doing a git checkout to the 1.1.1 stable version

   ...
    git clone https://github.com/openssl/openssl.git && \
    cd openssl && \
    git checkout OpenSSL_1_1_1-stable && \
    ./config \
    ...

Looks like unbound currently can't work with newer versions of openssl on older architecture.