i586-unknown-linux-gnu target generates binaries containing Intel CET opcodes which are illegal on i586 processors

[teknoman117]

As the title says, I generated an i586 rust toolchain by doing the following:

git clone https://github.com/rust-lang/rust
cd rust
git checkout 1.58.0
git submodule update --init --recursive
./x.py setup --build i686-unknown-linux-gnu --host i586-unknown-linux-gnu --target i586-unknown-linux-gnu -j 32
./x.py dist --build i686-unknown-linux-gnu --host i586-unknown-linux-gnu --target i586-unknown-linux-gnu -j 32
# transfer build/dist/rust-1.58.0-dev-i586-unknown-linux-gnu.tar.xz to an i586 target

However, after installing the toolchain, attempting to run any of the rust tooling throws an illegal instruction exception.

$ uname -a
Linux remembrance 5.16.0-gentoo #8 SMP PREEMPT Thu Jan 13 18:49:55 PST 2022 i586 AMD-K6(tm) 3D processor AuthenticAMD GNU/Linux
$ cargo new hello
Illegal instruction
$ rustc -vV
Illegal instruction

Running rustc under gdb and looking at the failing instruction shows that the binary contains the "endbr32" instruction, which is part of the Intel CET instruction set on Tiger Lake CPUs. These decode as "NOP" on i686 processors and x86_64 CPUs, but they are illegal opcodes for i586 processors. While I recognize that this processor is ancient, there is still one i586 processor series in production - the Vortex86. For any current Linux distros which still support i586, polkit depends on spidermonkey, which is partially written in rust.

$ gdb rustc
GNU gdb (Gentoo 11.1 vanilla) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "i586-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from rustc...
(No debugging symbols found in rustc)
(gdb) r
Starting program: /usr/bin/rustc
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0xadc94000 in ?? () from /opt/rust-bin-1.58.0/bin/../lib/libstd-89b7c7054cbe007e.so
(gdb) x/i $pc
=> 0xadc94000:	endbr32
(gdb)

Meta

rustc --version --verbose: (from a chroot on an x86_64 desktop)

rustc 1.58.0-dev
binary: rustc
commit-hash: unknown
commit-date: unknown
host: i586-unknown-linux-gnu
release: 1.58.0-dev
LLVM version: 13.0.0

[Urgau]

you may want to look here #82435, it's not exactly the same issue but some informations may be relevant to you

[teknoman117]

I was just a little surprised over this considering the vendoring for i586 puts -march=pentium into the mix of cflags. I'm going to try adding -fcf-protection=none to the cflags and see what happens. I didn't think rust actually supported CET opcodes yet considering the ticket is still open.

[teknoman117]

I was just a little surprised over this considering the vendoring for i586 puts -march=pentium into the mix of cflags. I'm going to try adding -fcf-protection=none to the cflags and see what happens. I didn't think rust actually supported CET opcodes yet considering the ticket is still open.

Adding -fcf-protection=none to my compilation flags did indeed work. This K6-2/500 system can now compile rust projects ~520 times slower than my Threadripper 1950X desktop XD

[llvm]
cflags = "-fcf-protection=none"
cxxflags = "-fcf-protection=none"
ldflags = "-fcf-protection=none"

[cuviper]

I'm curious to know where the apparently-default -fcf-protection came from in your build!

I can now compile rust projects hundreds of times slower than my Threadripper 1950X desktop XD

🐢

[teknoman117]

I'm curious to know where the apparently-default -fcf-protection came from in your build!

I can now compile rust projects hundreds of times slower than my Threadripper 1950X desktop XD

turtle

I used the i386/ubuntu:20.04 docker container to do the build in order to isolate Rust from my host environment (presumably). (rustbuild is just a snapshot of the container I made with all of the required build tooling installed)

$ docker run --rm -it -v /home/nlewis/Sources:/home/nlewis/Sources i386/rustbuild:20.04 /bin/bash --login
WARNING: The requested image's platform (linux/386) does not match the detected host platform (linux/amd64) and no specific platform was requested
root@0325459fba92:/# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/i686-linux-gnu/9/lto-wrapper
Target: i686-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-17ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=i686-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-targets=all --enable-multiarch --disable-werror --with-arch-32=i686 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=i686-linux-gnu --host=i686-linux-gnu --target=i686-linux-gnu
Thread model: posix
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
root@0325459fba92:/#

[gyakovlev]

in gentoo we do some stuff with -fcf-protection but certainly not for gcc version used here and not for i586 and not by default, it's an opt-in thing. but we'll double check on our side.

[teknoman117]

in gentoo we do some stuff with -fcf-protection but certainly not for gcc version used here and not for i586 and not by default, it's an opt-in thing. but we'll double check on our side.

I didn't bump into any other packages having problems with -fcf-protection in the whole build. They only ended up in rust and things compiled by rust. I was curious if LLVM might be silently enabling it by default because Intel explicitly chose opcodes that were NOPs on older CPUs (well, as old as i686 anyways).

[gyakovlev]

yeah I checked specs and we don't even enable it on x86 at all, only x86_64 and only if USE=cet.
since we will probably have i586 boostrap tarball soonish I'll keep in mind to disable fcf explicitly in the ebuild.