Use rustc directly instead of cargo

This is a big PR, but most of it is interdependent to the rest.

  - Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
    `liballoc`, `libcompiler_builtins`.

      + The Rust modules are now much smaller since they do not contain
        several copies of those libraries. Our example `.ko` on release
        is just 12 KiB, down from 1.3 MiB. For reference:

            `vmlinux` on release w/  Rust is 23 MiB (compressed: 2.1 MiB)
            `vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)

        i.e. the bulk is now shared.

      + Multiple builtin modules are now supported since their symbols
        do not collide against each other (fixes #9).

      + Faster compilation (less crates to compile & less repetition).

      + We achieve this by compiling all the shared code to `.rlib`s
        (and the `.so` for the proc macro). For loadable modules,
        we need to rely on the upcoming v0 Rust mangling scheme,
        plus we need to export the Rust symbols needed by the `.ko`s.

  - Simpler, flat file structure: now a small driver may only need
    a single file like `drivers/char/rust_example.rs`, like in C.

      + All the `rust/*` and `driver/char/rust_example/*` files moved
        to fit in the new structure: less files around.

  - Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.

      + Also helps with the faster compilation.

  - Dependency handling integration with `Kbuild`/`fixdep`.

      + Changes to the Rust standard library, kernel headers (bindings),
        `rust/` source files, `.rs` changes, command-line changes,
        flag changes, etc. all trigger recompilation as needed.

      + Works as expected with parallel support (`-j`).

  - Automatic generation of the `exports.c` list:

      + Instead of manually handling the list, all non-local functions
        available in `core`, `alloc` and `kernel` are exported, so all
        modules should work, regardless of what they need, and without
        failing linking due to symbols in the manual list not existing
        (e.g. due to differences in config options).

      + They are a lot, though:

          * ~6k Rust symbols vs. ~4k C symbols in release.

          * However, 4k of those are `bindings_raw` (i.e. duplicated C
            ones), which shouldn't be exported. Thus we should look
            into making `bindings_raw` private to the crate (at the
            moment, the (first) Rust example requires
            `<kernel::bindings...::miscdevice as Default>::default`).

      + Licensing:

          * `kernel`'s symbols are exported as GPL.

          * `core`'s and `alloc`'s symbols are exported as non-GPL so
            that third-parties can build Rust modules as long as they
            write their own kernel support infrastructure, i.e. without
            taking advantage of `kernel`. This seemed to make the most
            sense compared to other exports from the kernel, plus it
            follows more closely the original licence of the crates.

  - Support for GCC-compiled kernels:

    + The generated bindings do not have meaningful differences in our
      release config, between GCC 10.1 and Clang 11.

    + Other configs (e.g. our debug one) may add/remove types and functions.
      That is fine unless we use them form our bindings.

    + However, there are config options that may not work (e.g.
      the randstruct GCC plugin if we use one of those structs).

  - Support for `arm64` architecture:

    + Added to the CI: BusyBox is cross-compiled on the fly (increased
      timeout a bit to match).

    + Requires weakening of a few compiler builtins and adding
      `copy_{from,to}_user` helpers.

  - Proper `make clean` support.

  - Offline builds by default (there is no "online compilation" anymore;
    fixes #17).

  - No interleaved Cargo output (fixes #29).

  - No nightly dependency on Cargo's `build-std`; since now we manage
    the cross-compilation ourselves (should fix #27).

  - "Big" kallsyms symbol support:

    + I already raised ksym names from 128 to 256 back when I wrote the first
      integration. However, Rust symbols can be huge in debug/non-optimized,
      so I increased it again to 512; plus the module name from 56 to 248.

    + In turn, this required tuning the table format to support 2-byte lengths
      for ksyms. Compression at generation and kernel decompression is covered,
      although it may be the case that some script/tool also requires changes
      to understand the new table format.

  - Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
    is added to enable/disable it manually, regardless of whether one has
    `rustc` available or not (`CONFIG_HAS_RUST`).

  - Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
    by default following what the user selected for C, but customizable
    through a Kconfig menu. As well as options for tweaking overflow
    checks and debug assertions.

  - This rewrite of the Kbuild support is cleaner, i.e. less hacks
    in general handling paths (e.g. no more `shell readlink` for `O=`).

  - Duplicated the example driver 3 times so that we can test in the CI
    that 2 builtins and 2 loadables work, all at the same time.

  - Do not export any helpers' symbols.

  - Updated the quick start guide.

  - Updated CI:

      + Now we always test with 2 builtins and 2 loadables Rust example
        drivers, removing the matrix test for builtin/loadable.

      + Added `toolchain` to matrix: now we test building with GCC,
        Clang or a full LLVM toolchain.

      + Added `arch` to matrix: now we test both arm64 and x86_64.

      + Added `rustc` to matrix: now we test with a very recent nightly
        as well.

      + Only build `output == build` once to reduce the number
        of combinations.

      + Debug x86_64 config: more things enabled (debuginfo, kgdb,
        unit testing, etc.) that mimic more what a developer would have.
        Running the CI will be slightly slower, but should be OK.
        Also enable `-C opt-level=0` to test that such an extreme works
        and also to see how much bloated everything becomes.

      + Release x86_64 config: disabled `EXPERT` and changed a few things
        to make it look more like a normal desktop configuration,
        although it is still pretty minimal.

      + The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
        very minimal, for the particular CPU we are simulating.

      + Update configs to v5.10.

      + Use `$GITHUB_ENV` to simplify.

  - Less `extern crate`s needed since we pass it via `rustc`
    (closer to idiomatic 2018 edition Rust code).

Things to note:

  - There is two more nightly features used:

      + The new Rust mangling scheme: we know it will be stable
        (and the default on, later on).

      + The binary dep-info output: if we remove all other nightly
        features, this one can easily go too.

  - The hack at `exports.c` to export symbols to loadable modules.

  - The hack at `allocator.rs` to get the `__rust_*()` functions.

  - The hack to get the proper flags for bindgen on GCC builds.

Signed-off-by: Miguel Ojeda <ojeda@kernel.org>

Author: ojeda

.github/workflows/busybox.config

@@ -6,74 +6,172 @@ on:
 jobs:
   ci:
     runs-on: ubuntu-20.04
-    timeout-minutes: 15
+    timeout-minutes: 20
 
     strategy:
       matrix:
-        mode: [debug, release]
-        module: [builtin, loadable]
-        outputdir: [src, build]
+        arch: [x86_64, arm64]
+        toolchain: [gcc, clang, llvm]
+        config: [debug, release]
+        rustc: [2021-01-02]
+        output: [src] # [src, build]
+        install: [rustup] # [rustup, standalone]
+        sysroot: [common] # [common, custom]
+
+        # A few independent combinations to avoid exploding the matrix:
+        #   - The other option for `output`.
+        #   - Different nightlies for `rustc`.
+        #   - The other three (`install`, `sysroot`) combinations
+        #     (they are interrelated, so the cross-product needs to be tested)
+        include:
+          - arch: x86_64
+            toolchain: gcc
+            config: debug
+            rustc: 2021-01-09
+            output: build
+            install: rustup
+            sysroot: custom
+
+          - arch: arm64
+            toolchain: clang
+            config: release
+            rustc: 2021-01-16
+            output: build
+            install: standalone
+            sysroot: common
+
+          - arch: x86_64
+            toolchain: llvm
+            config: debug
+            rustc: 2021-01-19
+            output: build
+            install: standalone
+            sysroot: custom
 
     steps:
-      # Setup
+      # Setup: checkout
       - uses: actions/checkout@v2
-      - run: wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
+
+      # Setup: variables
+      - if: matrix.arch == 'x86_64'
+        run: |
+          echo 'IMAGE_PATH=arch/x86/boot/bzImage' >> $GITHUB_ENV
+          echo 'QEMU_ARCH=x86_64' >> $GITHUB_ENV
+          echo 'QEMU_MACHINE=pc' >> $GITHUB_ENV
+          echo 'QEMU_CPU=Cascadelake-Server' >> $GITHUB_ENV
+          echo 'QEMU_APPEND=console=ttyS0' >> $GITHUB_ENV
+      - if: matrix.arch == 'arm64'
+        run: |
+          echo 'MAKE_ARCH=ARCH=arm64' >> $GITHUB_ENV
+          echo 'MAKE_CROSS_COMPILE=CROSS_COMPILE=aarch64-linux-gnu-' >> $GITHUB_ENV
+          echo 'IMAGE_PATH=arch/arm64/boot/Image.gz' >> $GITHUB_ENV
+          echo 'QEMU_ARCH=aarch64' >> $GITHUB_ENV
+          echo 'QEMU_MACHINE=virt' >> $GITHUB_ENV
+          echo 'QEMU_CPU=cortex-a72' >> $GITHUB_ENV
+
+      - if: matrix.toolchain == 'clang'
+        run: echo 'MAKE_TOOLCHAIN=CC=clang-11' >> $GITHUB_ENV
+      - if: matrix.toolchain == 'llvm'
+        run: echo 'MAKE_TOOLCHAIN=LLVM=1' >> $GITHUB_ENV
+
+      - if: matrix.output == 'build'
+        run: |
+          echo 'MAKE_OUTPUT=O=build' >> $GITHUB_ENV
+          echo 'BUILD_DIR=build/' >> $GITHUB_ENV
+
+      - if: matrix.sysroot == 'custom'
+        run: |
+          echo 'RUSTC_SYSROOT=--sysroot=$HOME/sysroot' >> $GITHUB_ENV
+          echo "MAKE_SYSROOT=KRUSTCFLAGS=--sysroot=$HOME/sysroot" >> $GITHUB_ENV
+
+      # Setup: LLVM
+      - run: curl https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
       - run: sudo add-apt-repository 'deb http://apt.llvm.org/focal/ llvm-toolchain-focal-11 main'
       - run: sudo apt-get update -y
-      - run: sudo apt-get install -y clang-11 libelf-dev qemu-system-x86 busybox-static
-      - run: rustup default nightly-2021-01-02
-      - run: rustup component add rustfmt
-      - run: rustup component add rust-src
+      - run: sudo apt-get install -y llvm-11 clang-11 lld-11
+      - run: echo $(llvm-config-11 --bindir) >> $GITHUB_PATH
 
-      # Build
-      - run: cp .github/workflows/kernel-${{ matrix.mode }}.config .config
+      # Setup: GCC
+      - if: matrix.arch == 'arm64'
+        run: sudo apt-get install -y gcc-aarch64-linux-gnu
 
-      - if: matrix.module == 'loadable'
-        run: sed -i -E 's/^(CONFIG_RUST_EXAMPLE=)(y)$/\1m/g' .config
+      # Setup: libelf
+      - run: sudo apt-get install -y libelf-dev
 
-      - if: matrix.outputdir == 'build'
-        run: mkdir build && mv .config build/.config
+      # Setup: QEMU
+      - if: matrix.arch == 'x86_64'
+        run: sudo apt-get install -y qemu-system-x86
+      - if: matrix.arch == 'arm64'
+        run: sudo apt-get install -y qemu-system-arm
 
-      - if: matrix.outputdir == 'src'
-        run: make         CC=clang-11 LLVM_CONFIG_PATH=llvm-config-11 -j3
-      - if: matrix.outputdir == 'build'
-        run: make O=build CC=clang-11 LLVM_CONFIG_PATH=llvm-config-11 -j3
+      # Setup: rustc
+      - if: matrix.install == 'rustup'
+        run: |
+          rustup default nightly-${{ matrix.rustc }}
+          rustup component add rustfmt
+      - if: matrix.install == 'standalone'
+        run: |
+          curl https://static.rust-lang.org/dist/${{ matrix.rustc }}/rust-nightly-x86_64-unknown-linux-gnu.tar.gz | tar xz
+          rust-nightly-x86_64-unknown-linux-gnu/install.sh --without=rust-docs --prefix=$HOME/rustc
+          echo $HOME/rustc/bin >> $GITHUB_PATH
 
-      # Run
-      - if: matrix.module == 'builtin'
-        run: sed -i '/rust_example/d' .github/workflows/qemu-initramfs.desc
+      # Setup: rustc native libs
+      - if: matrix.sysroot == 'custom'
+        run: |
+          mkdir $(rustc ${{ env.RUSTC_SYSROOT }} --print sysroot)
+          ln -s $(rustc --print sysroot)/lib $(rustc ${{ env.RUSTC_SYSROOT }} --print sysroot)/lib
 
-      - if: matrix.outputdir == 'build'
-        run: sed -i 's:drivers/:build/drivers/:' .github/workflows/qemu-initramfs.desc
+      # Setup: rustc source
+      - if: matrix.install == 'rustup' && matrix.sysroot == 'common'
+        run: rustup component add rust-src
+      - if: matrix.install != 'rustup' || matrix.sysroot != 'common'
+        run: |
+          git clone -n https://github.com/rust-lang/rust $(rustc ${{ env.RUSTC_SYSROOT }} --print sysroot)/lib/rustlib/src/rust
+          cd $(rustc ${{ env.RUSTC_SYSROOT }} --print sysroot)/lib/rustlib/src/rust
+          git checkout $(rustc -vV | grep -F 'commit-hash' | awk '{print $2}')
+          git submodule update --init library
 
-      - if: matrix.outputdir == 'src'
-        run:       usr/gen_init_cpio .github/workflows/qemu-initramfs.desc > qemu-initramfs.img
-      - if: matrix.outputdir == 'build'
-        run: build/usr/gen_init_cpio .github/workflows/qemu-initramfs.desc > qemu-initramfs.img
+      # Setup: compiler-builtins source
+      - run: git clone --depth 1 -b 0.1.39 https://github.com/rust-lang/compiler-builtins.git $(rustc ${{ env.RUSTC_SYSROOT }} --print sysroot)/lib/rustlib/src/compiler-builtins
 
-      - if: matrix.outputdir == 'src'
-        run: qemu-system-x86_64 -kernel       arch/x86/boot/bzImage -initrd qemu-initramfs.img -cpu Cascadelake-Server -smp 2 -nographic -no-reboot -append "console=ttyS0 ${{ matrix.module == 'builtin' && 'rust_example.my_i32=123321' || '' }}" | tee qemu-stdout.log
-      - if: matrix.outputdir == 'build'
-        run: qemu-system-x86_64 -kernel build/arch/x86/boot/bzImage -initrd qemu-initramfs.img -cpu Cascadelake-Server -smp 2 -nographic -no-reboot -append "console=ttyS0 ${{ matrix.module == 'builtin' && 'rust_example.my_i32=123321' || '' }}" | tee qemu-stdout.log
+      # Setup: bindgen
+      - run: cargo install --version 0.56.0 bindgen
+
+      # Setup: busybox
+      - run: git clone --depth 1 -b 1_30_1 https://github.com/mirror/busybox
+      - run: mv .github/workflows/busybox.config busybox/.config
+      - run: cd busybox && make ${{ env.MAKE_CROSS_COMPILE }} -j3
+
+      # Build
+      - run: mv .github/workflows/kernel-${{ matrix.arch }}-${{ matrix.config }}.config .config
+
+      - if: matrix.output == 'build'
+        run: |
+          mkdir ${{ env.BUILD_DIR }}
+          mv .config ${{ env.BUILD_DIR }}.config
+          sed -i 's:drivers/:${{ env.BUILD_DIR }}drivers/:' .github/workflows/qemu-initramfs.desc
+
+      - run: make ${{ env.MAKE_ARCH }} ${{ env.MAKE_CROSS_COMPILE }} ${{ env.MAKE_TOOLCHAIN }} ${{ env.MAKE_OUTPUT }} ${{ env.MAKE_SYSROOT }} -j3
+
+      # Run
+      - run: ${{ env.BUILD_DIR }}usr/gen_init_cpio .github/workflows/qemu-initramfs.desc > qemu-initramfs.img
+
+      - run: qemu-system-${{ env.QEMU_ARCH }} -kernel ${{ env.BUILD_DIR }}${{ env.IMAGE_PATH }} -initrd qemu-initramfs.img -M ${{ env.QEMU_MACHINE }} -cpu ${{ env.QEMU_CPU }} -smp 2 -nographic -no-reboot -append '${{ env.QEMU_APPEND }} rust_example.my_i32=123321 rust_example_2.my_i32=234432' | tee qemu-stdout.log
 
       # Check
-      - run: grep -F 'Rust Example (init)' qemu-stdout.log
-      - run: "grep 'my_i32: \\+123321' qemu-stdout.log"
-      - if: matrix.module == 'loadable'
-        run: grep -F 'Rust Example (exit)' qemu-stdout.log
+      - run: grep -F     '] Rust Example (init)' qemu-stdout.log
+      - run: grep -F '] [2] Rust Example (init)' qemu-stdout.log
+      - run: grep -F '] [3] Rust Example (init)' qemu-stdout.log
+      - run: grep -F '] [4] Rust Example (init)' qemu-stdout.log
+
+      - run: "grep -F     ']   my_i32:   123321' qemu-stdout.log"
+      - run: "grep -F '] [2]   my_i32:   234432' qemu-stdout.log"
+      - run: "grep -F '] [3]   my_i32:   345543' qemu-stdout.log"
+      - run: "grep -F '] [4]   my_i32:   456654' qemu-stdout.log"
+
+      - run: grep -F '] [3] Rust Example (exit)' qemu-stdout.log
+      - run: grep -F '] [4] Rust Example (exit)' qemu-stdout.log
 
       # Report
-      - if: matrix.outputdir == 'src' && matrix.module == 'loadable'
-        run: ls -l       drivers/char/rust_example/rust_example.ko
-      - if: matrix.outputdir == 'build' && matrix.module == 'loadable'
-        run: ls -l build/drivers/char/rust_example/rust_example.ko
-
-      - if: matrix.outputdir == 'src'
-        run: ls -l       vmlinux       arch/x86/boot/bzImage
-      - if: matrix.outputdir == 'build'
-        run: ls -l build/vmlinux build/arch/x86/boot/bzImage
-
-      - if: matrix.outputdir == 'src'
-        run: size        vmlinux
-      - if: matrix.outputdir == 'build'
-        run: size  build/vmlinux
+      - run: ls -l ${{ env.BUILD_DIR }}drivers/char/rust_example.o ${{ env.BUILD_DIR }}drivers/char/rust_example_3.ko ${{ env.BUILD_DIR }}rust/*.o ${{ env.BUILD_DIR }}vmlinux ${{ env.BUILD_DIR }}${{ env.IMAGE_PATH }}
+      - run: size  ${{ env.BUILD_DIR }}drivers/char/rust_example.o ${{ env.BUILD_DIR }}drivers/char/rust_example_3.ko ${{ env.BUILD_DIR }}rust/*.o ${{ env.BUILD_DIR }}vmlinux