Makefile ui #29
Labels
• kbuild
Related to building the kernel, `make`, `Kbuild`, `Kconfig` options...
Comments
kloenk
added
rust-cargo
• kbuild
|
That was on purpose so that we have something similar to the |
ojeda
added a commit
that referenced
this issue
Dec 9, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure. Way less files around!
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Offline builds, always; i.e. there is no "online compilation"
anymore (fixes #17).
- No more interleaved Cargo output (fixes #29).
- One less nightly dependency (Cargo's `build-std`); since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking `rustc` options, like overflow checks.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is one more nightly feature used (the new Rust mangling scheme),
but we know that one will be stable (and the default one, later on).
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
There are a few TODOs that we can improve later if we agree on this:
- Kbuild:
+ Actually use the `*.d` files.
+ Complete `make clean`.
+ Support single-object compilation.
+ Pass `objtool` to make the ORC unwinder work.
+ Echo the building of the rust/* libraries and the bindgen call.
- Figure out how to pick symbols to export automatically from
Rust code instead of managing the list by hand.
Perhaps we could use a no-op macro on the Rust code, which is then
parse by a script to pick up the symbols:
pub fn foo() {}
export_symbol!(foo);
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 9, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure. Way less files around!
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Offline builds, always; i.e. there is no "online compilation"
anymore (fixes #17).
- No more interleaved Cargo output (fixes #29).
- One less nightly dependency (Cargo's `build-std`); since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking `rustc` options, like overflow checks.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is one more nightly feature used (the new Rust mangling scheme),
but we know that one will be stable (and the default one, later on).
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
There are a few TODOs that we can improve later if we agree on this:
- Kbuild:
+ Actually use the `*.d` files.
+ Complete `make clean`.
+ Support single-object compilation.
+ Pass `objtool` to make the ORC unwinder work.
+ Echo the building of the rust/* libraries and the bindgen call.
- Figure out how to pick symbols to export automatically from
Rust code instead of managing the list by hand.
Perhaps we could use a no-op macro on the Rust code, which is then
parse by a script to pick up the symbols:
pub fn foo() {}
export_symbol!(foo);
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 9, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure. Way less files around!
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Offline builds, always; i.e. there is no "online compilation"
anymore (fixes #17).
- No more interleaved Cargo output (fixes #29).
- One less nightly dependency (Cargo's `build-std`); since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking `rustc` options, like overflow checks.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is one more nightly feature used (the new Rust mangling scheme),
but we know that one will be stable (and the default one, later on).
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
There are a few TODOs that we can improve later if we agree on this:
- Kbuild:
+ Actually use the `*.d` files.
+ Complete `make clean`.
+ Support single-object compilation.
+ Pass `objtool` to make the ORC unwinder work.
+ Echo the building of the rust/* libraries and the bindgen call.
- Figure out how to pick symbols to export automatically from
Rust code instead of managing the list by hand.
Perhaps we could use a no-op macro on the Rust code, which is then
parse by a script to pick up the symbols:
pub fn foo() {}
export_symbol!(foo);
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 9, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure. Way less files around!
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Offline builds, always; i.e. there is no "online compilation"
anymore (fixes #17).
- No more interleaved Cargo output (fixes #29).
- One less nightly dependency (Cargo's `build-std`); since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking `rustc` options, like overflow checks.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is one more nightly feature used (the new Rust mangling scheme),
but we know that one will be stable (and the default one, later on).
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
There are a few TODOs that we can improve later if we agree on this:
- Kbuild:
+ Actually use the `*.d` files.
+ Complete `make clean`.
+ Support single-object compilation.
+ Pass `objtool` to make the ORC unwinder work.
+ Echo the building of the rust/* libraries and the bindgen call.
- Figure out how to pick symbols to export automatically from
Rust code instead of managing the list by hand.
Perhaps we could use a no-op macro on the Rust code, which is then
parse by a script to pick up the symbols:
pub fn foo() {}
export_symbol!(foo);
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 12, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, etc. all trigger
recompilation of the proper things.
+ Works as expected with parallel support (`-j`).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No more interleaved Cargo output (fixes #29).
- One less nightly dependency (Cargo's `build-std`); since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking `rustc` options, like overflow checks.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is one more nightly feature used (the new Rust mangling scheme),
but we know that one will be stable (and the default one, later on).
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 16, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking relevant `rustc` options, like overflow
checks, debug assertions, optimization level, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
pushed a commit
that referenced
this issue
Dec 16, 2020
This patch fixes issue introduced by a previous commit where iWARP doorbell address wasn't initialized, causing call trace when any RDMA application wants to use this interface: Illegal doorbell address: 0000000000000000. Legal range for doorbell addresses is [0000000011431e08..00000000ec3799d3] WARNING: CPU: 11 PID: 11990 at drivers/net/ethernet/qlogic/qed/qed_dev.c:93 qed_db_rec_sanity.isra.12+0x48/0x70 [qed] ... hpsa scsi_transport_sas [last unloaded: crc8] CPU: 11 PID: 11990 Comm: rping Tainted: G S 5.10.0-rc1 #29 Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 01/22/2018 RIP: 0010:qed_db_rec_sanity.isra.12+0x48/0x70 [qed] ... RSP: 0018:ffffafc28458fa88 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffff8d0d4c620000 RCX: 0000000000000000 RDX: ffff8d10afde7d50 RSI: ffff8d10afdd8b40 RDI: ffff8d10afdd8b40 RBP: ffffafc28458fe38 R08: 0000000000000003 R09: 0000000000007fff R10: 0000000000000001 R11: ffffafc28458f888 R12: 0000000000000000 R13: 0000000000000000 R14: ffff8d0d43ccbbd0 R15: ffff8d0d48dae9c0 FS: 00007fbd5267e740(0000) GS:ffff8d10afdc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fbd4f258fb8 CR3: 0000000108d96003 CR4: 00000000001706e0 Call Trace: qed_db_recovery_add+0x6d/0x1f0 [qed] qedr_create_user_qp+0x57e/0xd30 [qedr] qedr_create_qp+0x5f3/0xab0 [qedr] ? lookup_get_idr_uobject.part.12+0x45/0x90 [ib_uverbs] create_qp+0x45d/0xb30 [ib_uverbs] ? ib_uverbs_cq_event_handler+0x30/0x30 [ib_uverbs] ib_uverbs_create_qp+0xb9/0xe0 [ib_uverbs] ib_uverbs_write+0x3f9/0x570 [ib_uverbs] ? security_mmap_file+0x62/0xe0 vfs_write+0xb7/0x200 ksys_write+0xaf/0xd0 ? syscall_trace_enter.isra.25+0x152/0x200 do_syscall_64+0x2d/0x40 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Fixes: 06e8d1d ("RDMA/qedr: Add support for user mode XRC-SRQ's") Link: https://lore.kernel.org/r/20201127163251.14533-1-palok@marvell.com Signed-off-by: Michal Kalderon <mkalderon@marvell.com> Signed-off-by: Igor Russkikh <irusskikh@marvell.com> Signed-off-by: Alok Prasad <palok@marvell.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
ojeda
added a commit
that referenced
this issue
Dec 16, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking relevant `rustc` options, like overflow
checks, debug assertions, optimization level, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Dec 28, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Jan 2, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Jan 3, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Jan 10, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Jan 10, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Jan 16, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Jan 19, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
added a commit
that referenced
this issue
Jan 19, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Support for custom `--sysroot` via `KRUSTCFLAGS`.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
ojeda
pushed a commit
that referenced
this issue
Feb 28, 2022
The trace_hardirqs_{on,off}() require the caller to setup frame pointer
properly. This because these two functions use macro 'CALLER_ADDR1' (aka.
__builtin_return_address(1)) to acquire caller info. If the $fp is used
for other purpose, the code generated this macro (as below) could trigger
memory access fault.
0xffffffff8011510e <+80>: ld a1,-16(s0)
0xffffffff80115112 <+84>: ld s2,-8(a1) # <-- paging fault here
The oops message during booting if compiled with 'irqoff' tracer enabled:
[ 0.039615][ T0] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000f8
[ 0.041925][ T0] Oops [#1]
[ 0.042063][ T0] Modules linked in:
[ 0.042864][ T0] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.17.0-rc1-00233-g9a20c48d1ed2 #29
[ 0.043568][ T0] Hardware name: riscv-virtio,qemu (DT)
[ 0.044343][ T0] epc : trace_hardirqs_on+0x56/0xe2
[ 0.044601][ T0] ra : restore_all+0x12/0x6e
[ 0.044721][ T0] epc : ffffffff80126a5c ra : ffffffff80003b94 sp : ffffffff81403db0
[ 0.044801][ T0] gp : ffffffff8163acd8 tp : ffffffff81414880 t0 : 0000000000000020
[ 0.044882][ T0] t1 : 0098968000000000 t2 : 0000000000000000 s0 : ffffffff81403de0
[ 0.044967][ T0] s1 : 0000000000000000 a0 : 0000000000000001 a1 : 0000000000000100
[ 0.045046][ T0] a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000
[ 0.045124][ T0] a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000054494d45
[ 0.045210][ T0] s2 : ffffffff80003b94 s3 : ffffffff81a8f1b0 s4 : ffffffff80e27b50
[ 0.045289][ T0] s5 : ffffffff81414880 s6 : ffffffff8160fa00 s7 : 00000000800120e8
[ 0.045389][ T0] s8 : 0000000080013100 s9 : 000000000000007f s10: 0000000000000000
[ 0.045474][ T0] s11: 0000000000000000 t3 : 7fffffffffffffff t4 : 0000000000000000
[ 0.045548][ T0] t5 : 0000000000000000 t6 : ffffffff814aa368
[ 0.045620][ T0] status: 0000000200000100 badaddr: 00000000000000f8 cause: 000000000000000d
[ 0.046402][ T0] [<ffffffff80003b94>] restore_all+0x12/0x6e
This because the $fp(aka. $s0) register is not used as frame pointer in the
assembly entry code.
resume_kernel:
REG_L s0, TASK_TI_PREEMPT_COUNT(tp)
bnez s0, restore_all
REG_L s0, TASK_TI_FLAGS(tp)
andi s0, s0, _TIF_NEED_RESCHED
beqz s0, restore_all
call preempt_schedule_irq
j restore_all
To fix above issue, here we add one extra level wrapper for function
trace_hardirqs_{on,off}() so they can be safely called by low level entry
code.
Signed-off-by: Changbin Du <changbin.du@gmail.com>
Fixes: 3c46979 ("riscv: Enable LOCKDEP_SUPPORT & fixup TRACE_IRQFLAGS_SUPPORT")
Cc: stable@vger.kernel.org
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
ojeda
pushed a commit
that referenced
this issue
Dec 4, 2022
The following has been observed when running stressng mmap since commit b653db7 ("mm: Clear page->private when splitting or migrating a page") watchdog: BUG: soft lockup - CPU#75 stuck for 26s! [stress-ng:9546] CPU: 75 PID: 9546 Comm: stress-ng Tainted: G E 6.0.0-revert-b653db77-fix+ #29 0357d79b60fb09775f678e4f3f64ef0579ad1374 Hardware name: SGI.COM C2112-4GP3/X10DRT-P-Series, BIOS 2.0a 05/09/2016 RIP: 0010:xas_descend+0x28/0x80 Code: cc cc 0f b6 0e 48 8b 57 08 48 d3 ea 83 e2 3f 89 d0 48 83 c0 04 48 8b 44 c6 08 48 89 77 18 48 89 c1 83 e1 03 48 83 f9 02 75 08 <48> 3d fd 00 00 00 76 08 88 57 12 c3 cc cc cc cc 48 c1 e8 02 89 c2 RSP: 0018:ffffbbf02a2236a8 EFLAGS: 00000246 RAX: ffff9cab7d6a0002 RBX: ffffe04b0af88040 RCX: 0000000000000002 RDX: 0000000000000030 RSI: ffff9cab60509b60 RDI: ffffbbf02a2236c0 RBP: 0000000000000000 R08: ffff9cab60509b60 R09: ffffbbf02a2236c0 R10: 0000000000000001 R11: ffffbbf02a223698 R12: 0000000000000000 R13: ffff9cab4e28da80 R14: 0000000000039c01 R15: ffff9cab4e28da88 FS: 00007fab89b85e40(0000) GS:ffff9cea3fcc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fab84e00000 CR3: 00000040b73a4003 CR4: 00000000003706e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <TASK> xas_load+0x3a/0x50 __filemap_get_folio+0x80/0x370 ? put_swap_page+0x163/0x360 pagecache_get_page+0x13/0x90 __try_to_reclaim_swap+0x50/0x190 scan_swap_map_slots+0x31e/0x670 get_swap_pages+0x226/0x3c0 folio_alloc_swap+0x1cc/0x240 add_to_swap+0x14/0x70 shrink_page_list+0x968/0xbc0 reclaim_page_list+0x70/0xf0 reclaim_pages+0xdd/0x120 madvise_cold_or_pageout_pte_range+0x814/0xf30 walk_pgd_range+0x637/0xa30 __walk_page_range+0x142/0x170 walk_page_range+0x146/0x170 madvise_pageout+0xb7/0x280 ? asm_common_interrupt+0x22/0x40 madvise_vma_behavior+0x3b7/0xac0 ? find_vma+0x4a/0x70 ? find_vma+0x64/0x70 ? madvise_vma_anon_name+0x40/0x40 madvise_walk_vmas+0xa6/0x130 do_madvise+0x2f4/0x360 __x64_sys_madvise+0x26/0x30 do_syscall_64+0x5b/0x80 ? do_syscall_64+0x67/0x80 ? syscall_exit_to_user_mode+0x17/0x40 ? do_syscall_64+0x67/0x80 ? syscall_exit_to_user_mode+0x17/0x40 ? do_syscall_64+0x67/0x80 ? do_syscall_64+0x67/0x80 ? common_interrupt+0x8b/0xa0 entry_SYSCALL_64_after_hwframe+0x63/0xcd The problem can be reproduced with the mmtests config config-workload-stressng-mmap. It does not always happen and when it triggers is variable but it has happened on multiple machines. The intent of commit b653db7 patch was to avoid the case where PG_private is clear but folio->private is not-NULL. However, THP tail pages uses page->private for "swp_entry_t if folio_test_swapcache()" as stated in the documentation for struct folio. This patch only clobbers page->private for tail pages if the head page was not in swapcache and warns once if page->private had an unexpected value. Link: https://lkml.kernel.org/r/20221019134156.zjyyn5aownakvztf@techsingularity.net Fixes: b653db7 ("mm: Clear page->private when splitting or migrating a page") Signed-off-by: Mel Gorman <mgorman@techsingularity.net> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Mel Gorman <mgorman@techsingularity.net> Cc: Yang Shi <shy828301@gmail.com> Cc: Brian Foster <bfoster@redhat.com> Cc: Dan Streetman <ddstreet@ieee.org> Cc: Miaohe Lin <linmiaohe@huawei.com> Cc: Oleksandr Natalenko <oleksandr@natalenko.name> Cc: Seth Jennings <sjenning@redhat.com> Cc: Vitaly Wool <vitaly.wool@konsulko.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
ojeda
pushed a commit
that referenced
this issue
Apr 29, 2024
As previously explained, the rehash delayed work migrates filters from one region to another. This is done by iterating over all chunks (all the filters with the same priority) in the region and in each chunk iterating over all the filters. When the work runs out of credits it stores the current chunk and entry as markers in the per-work context so that it would know where to resume the migration from the next time the work is scheduled. Upon error, the chunk marker is reset to NULL, but without resetting the entry markers despite being relative to it. This can result in migration being resumed from an entry that does not belong to the chunk being migrated. In turn, this will eventually lead to a chunk being iterated over as if it is an entry. Because of how the two structures happen to be defined, this does not lead to KASAN splats, but to warnings such as [1]. Fix by creating a helper that resets all the markers and call it from all the places the currently only reset the chunk marker. For good measures also call it when starting a completely new rehash. Add a warning to avoid future cases. [1] WARNING: CPU: 7 PID: 1076 at drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.c:407 mlxsw_afk_encode+0x242/0x2f0 Modules linked in: CPU: 7 PID: 1076 Comm: kworker/7:24 Tainted: G W 6.9.0-rc3-custom-00880-g29e61d91b77b #29 Hardware name: Mellanox Technologies Ltd. MSN3700/VMOD0005, BIOS 5.11 01/06/2019 Workqueue: mlxsw_core mlxsw_sp_acl_tcam_vregion_rehash_work RIP: 0010:mlxsw_afk_encode+0x242/0x2f0 [...] Call Trace: <TASK> mlxsw_sp_acl_atcam_entry_add+0xd9/0x3c0 mlxsw_sp_acl_tcam_entry_create+0x5e/0xa0 mlxsw_sp_acl_tcam_vchunk_migrate_all+0x109/0x290 mlxsw_sp_acl_tcam_vregion_rehash_work+0x6c/0x470 process_one_work+0x151/0x370 worker_thread+0x2cb/0x3e0 kthread+0xd0/0x100 ret_from_fork+0x34/0x50 </TASK> Fixes: 6f9579d ("mlxsw: spectrum_acl: Remember where to continue rehash migration") Signed-off-by: Ido Schimmel <idosch@nvidia.com> Tested-by: Alexander Zubkov <green@qrator.net> Reviewed-by: Petr Machata <petrm@nvidia.com> Signed-off-by: Petr Machata <petrm@nvidia.com> Reviewed-by: Simon Horman <horms@kernel.org> Link: https://lore.kernel.org/r/cc17eed86b41dd829d39b07906fec074a9ce580e.1713797103.git.petrm@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ojeda
pushed a commit
that referenced
this issue
May 27, 2024
Reinitialize the whole EST structure would also reset the mutex lock which is embedded in the EST structure, and then trigger the following warning. To address this, move the lock to struct stmmac_priv. We also need to reacquire the mutex lock when doing this initialization. DEBUG_LOCKS_WARN_ON(lock->magic != lock) WARNING: CPU: 3 PID: 505 at kernel/locking/mutex.c:587 __mutex_lock+0xd84/0x1068 Modules linked in: CPU: 3 PID: 505 Comm: tc Not tainted 6.9.0-rc6-00053-g0106679839f7-dirty #29 Hardware name: NXP i.MX8MPlus EVK board (DT) pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : __mutex_lock+0xd84/0x1068 lr : __mutex_lock+0xd84/0x1068 sp : ffffffc0864e3570 x29: ffffffc0864e3570 x28: ffffffc0817bdc78 x27: 0000000000000003 x26: ffffff80c54f1808 x25: ffffff80c9164080 x24: ffffffc080d723ac x23: 0000000000000000 x22: 0000000000000002 x21: 0000000000000000 x20: 0000000000000000 x19: ffffffc083bc3000 x18: ffffffffffffffff x17: ffffffc08117b080 x16: 0000000000000002 x15: ffffff80d2d40000 x14: 00000000000002da x13: ffffff80d2d404b8 x12: ffffffc082b5a5c8 x11: ffffffc082bca680 x10: ffffffc082bb2640 x9 : ffffffc082bb2698 x8 : 0000000000017fe8 x7 : c0000000ffffefff x6 : 0000000000000001 x5 : ffffff8178fe0d48 x4 : 0000000000000000 x3 : 0000000000000027 x2 : ffffff8178fe0d50 x1 : 0000000000000000 x0 : 0000000000000000 Call trace: __mutex_lock+0xd84/0x1068 mutex_lock_nested+0x28/0x34 tc_setup_taprio+0x118/0x68c stmmac_setup_tc+0x50/0xf0 taprio_change+0x868/0xc9c Fixes: b2aae65 ("net: stmmac: add mutex lock to protect est parameters") Signed-off-by: Xiaolei Wang <xiaolei.wang@windriver.com> Reviewed-by: Simon Horman <horms@kernel.org> Reviewed-by: Serge Semin <fancer.lancer@gmail.com> Reviewed-by: Andrew Halaney <ahalaney@redhat.com> Link: https://lore.kernel.org/r/20240513014346.1718740-2-xiaolei.wang@windriver.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ojeda
pushed a commit
that referenced
this issue
Jun 11, 2024
cpumask_of_node() can be called for NUMA_NO_NODE inside do_map_benchmark() resulting in the following sanitizer report: UBSAN: array-index-out-of-bounds in ./arch/x86/include/asm/topology.h:72:28 index -1 is out of range for type 'cpumask [64][1]' CPU: 1 PID: 990 Comm: dma_map_benchma Not tainted 6.9.0-rc6 #29 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) Call Trace: <TASK> dump_stack_lvl (lib/dump_stack.c:117) ubsan_epilogue (lib/ubsan.c:232) __ubsan_handle_out_of_bounds (lib/ubsan.c:429) cpumask_of_node (arch/x86/include/asm/topology.h:72) [inline] do_map_benchmark (kernel/dma/map_benchmark.c:104) map_benchmark_ioctl (kernel/dma/map_benchmark.c:246) full_proxy_unlocked_ioctl (fs/debugfs/file.c:333) __x64_sys_ioctl (fs/ioctl.c:890) do_syscall_64 (arch/x86/entry/common.c:83) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130) Use cpumask_of_node() in place when binding a kernel thread to a cpuset of a particular node. Note that the provided node id is checked inside map_benchmark_ioctl(). It's just a NUMA_NO_NODE case which is not handled properly later. Found by Linux Verification Center (linuxtesting.org). Fixes: 65789da ("dma-mapping: add benchmark support for streaming DMA APIs") Signed-off-by: Fedor Pchelkin <pchelkin@ispras.ru> Acked-by: Barry Song <baohua@kernel.org> Signed-off-by: Christoph Hellwig <hch@lst.de>
Darksonn
pushed a commit
to Darksonn/linux
that referenced
this issue
Jan 17, 2025
syzbot reports that a recent fix causes nesting issues between the (now) raw timeoutlock and the eventfd locking: ============================= [ BUG: Invalid wait context ] 6.13.0-rc4-00080-g9828a4c0901f Rust-for-Linux#29 Not tainted ----------------------------- kworker/u32:0/68094 is trying to lock: ffff000014d7a520 (&ctx->wqh#2){..-.}-{3:3}, at: eventfd_signal_mask+0x64/0x180 other info that might help us debug this: context-{5:5} 6 locks held by kworker/u32:0/68094: #0: ffff0000c1d98148 ((wq_completion)iou_exit){+.+.}-{0:0}, at: process_one_work+0x4e8/0xfc0 Rust-for-Linux#1: ffff80008d927c78 ((work_completion)(&ctx->exit_work)){+.+.}-{0:0}, at: process_one_work+0x53c/0xfc0 Rust-for-Linux#2: ffff0000c59bc3d8 (&ctx->completion_lock){+.+.}-{3:3}, at: io_kill_timeouts+0x40/0x180 Rust-for-Linux#3: ffff0000c59bc358 (&ctx->timeout_lock){-.-.}-{2:2}, at: io_kill_timeouts+0x48/0x180 Rust-for-Linux#4: ffff800085127aa0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire+0x8/0x38 Rust-for-Linux#5: ffff800085127aa0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire+0x8/0x38 stack backtrace: CPU: 7 UID: 0 PID: 68094 Comm: kworker/u32:0 Not tainted 6.13.0-rc4-00080-g9828a4c0901f Rust-for-Linux#29 Hardware name: linux,dummy-virt (DT) Workqueue: iou_exit io_ring_exit_work Call trace: show_stack+0x1c/0x30 (C) __dump_stack+0x24/0x30 dump_stack_lvl+0x60/0x80 dump_stack+0x14/0x20 __lock_acquire+0x19f8/0x60c8 lock_acquire+0x1a4/0x540 _raw_spin_lock_irqsave+0x90/0xd0 eventfd_signal_mask+0x64/0x180 io_eventfd_signal+0x64/0x108 io_req_local_work_add+0x294/0x430 __io_req_task_work_add+0x1c0/0x270 io_kill_timeout+0x1f0/0x288 io_kill_timeouts+0xd4/0x180 io_uring_try_cancel_requests+0x2e8/0x388 io_ring_exit_work+0x150/0x550 process_one_work+0x5e8/0xfc0 worker_thread+0x7ec/0xc80 kthread+0x24c/0x300 ret_from_fork+0x10/0x20 because after the preempt-rt fix for the timeout lock nesting inside the io-wq lock, we now have the eventfd spinlock nesting inside the raw timeout spinlock. Rather than play whack-a-mole with other nesting on the timeout lock, split the deletion and killing of timeouts so queueing the task_work for the timeout cancelations can get done outside of the timeout lock. Reported-by: syzbot+b1fc199a40b65d601b65@syzkaller.appspotmail.com Fixes: 020b40f ("io_uring: make ctx->timeout_lock a raw spinlock") Signed-off-by: Jens Axboe <axboe@kernel.dk>
ojeda
pushed a commit
that referenced
this issue
Feb 4, 2025
KMSAN reported an uninit-value access in vxlan_vnifilter_dump() [1]. If the length of the netlink message payload is less than sizeof(struct tunnel_msg), vxlan_vnifilter_dump() accesses bytes beyond the message. This can lead to uninit-value access. Fix this by returning an error in such situations. [1] BUG: KMSAN: uninit-value in vxlan_vnifilter_dump+0x328/0x920 drivers/net/vxlan/vxlan_vnifilter.c:422 vxlan_vnifilter_dump+0x328/0x920 drivers/net/vxlan/vxlan_vnifilter.c:422 rtnl_dumpit+0xd5/0x2f0 net/core/rtnetlink.c:6786 netlink_dump+0x93e/0x15f0 net/netlink/af_netlink.c:2317 __netlink_dump_start+0x716/0xd60 net/netlink/af_netlink.c:2432 netlink_dump_start include/linux/netlink.h:340 [inline] rtnetlink_dump_start net/core/rtnetlink.c:6815 [inline] rtnetlink_rcv_msg+0x1256/0x14a0 net/core/rtnetlink.c:6882 netlink_rcv_skb+0x467/0x660 net/netlink/af_netlink.c:2542 rtnetlink_rcv+0x35/0x40 net/core/rtnetlink.c:6944 netlink_unicast_kernel net/netlink/af_netlink.c:1321 [inline] netlink_unicast+0xed6/0x1290 net/netlink/af_netlink.c:1347 netlink_sendmsg+0x1092/0x1230 net/netlink/af_netlink.c:1891 sock_sendmsg_nosec net/socket.c:711 [inline] __sock_sendmsg+0x330/0x3d0 net/socket.c:726 ____sys_sendmsg+0x7f4/0xb50 net/socket.c:2583 ___sys_sendmsg+0x271/0x3b0 net/socket.c:2637 __sys_sendmsg net/socket.c:2669 [inline] __do_sys_sendmsg net/socket.c:2674 [inline] __se_sys_sendmsg net/socket.c:2672 [inline] __x64_sys_sendmsg+0x211/0x3e0 net/socket.c:2672 x64_sys_call+0x3878/0x3d90 arch/x86/include/generated/asm/syscalls_64.h:47 do_syscall_x64 arch/x86/entry/common.c:52 [inline] do_syscall_64+0xd9/0x1d0 arch/x86/entry/common.c:83 entry_SYSCALL_64_after_hwframe+0x77/0x7f Uninit was created at: slab_post_alloc_hook mm/slub.c:4110 [inline] slab_alloc_node mm/slub.c:4153 [inline] kmem_cache_alloc_node_noprof+0x800/0xe80 mm/slub.c:4205 kmalloc_reserve+0x13b/0x4b0 net/core/skbuff.c:587 __alloc_skb+0x347/0x7d0 net/core/skbuff.c:678 alloc_skb include/linux/skbuff.h:1323 [inline] netlink_alloc_large_skb+0xa5/0x280 net/netlink/af_netlink.c:1196 netlink_sendmsg+0xac9/0x1230 net/netlink/af_netlink.c:1866 sock_sendmsg_nosec net/socket.c:711 [inline] __sock_sendmsg+0x330/0x3d0 net/socket.c:726 ____sys_sendmsg+0x7f4/0xb50 net/socket.c:2583 ___sys_sendmsg+0x271/0x3b0 net/socket.c:2637 __sys_sendmsg net/socket.c:2669 [inline] __do_sys_sendmsg net/socket.c:2674 [inline] __se_sys_sendmsg net/socket.c:2672 [inline] __x64_sys_sendmsg+0x211/0x3e0 net/socket.c:2672 x64_sys_call+0x3878/0x3d90 arch/x86/include/generated/asm/syscalls_64.h:47 do_syscall_x64 arch/x86/entry/common.c:52 [inline] do_syscall_64+0xd9/0x1d0 arch/x86/entry/common.c:83 entry_SYSCALL_64_after_hwframe+0x77/0x7f CPU: 0 UID: 0 PID: 30991 Comm: syz.4.10630 Not tainted 6.12.0-10694-gc44daa7e3c73 #29 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-3.fc41 04/01/2014 Fixes: f9c4bb0 ("vxlan: vni filtering support on collect metadata device") Reported-by: syzkaller <syzkaller@googlegroups.com> Signed-off-by: Shigeru Yoshida <syoshida@redhat.com> Reviewed-by: Ido Schimmel <idosch@nvidia.com> Link: https://patch.msgid.link/20250123145746.785768-1-syoshida@redhat.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ojeda
pushed a commit
that referenced
this issue
Feb 22, 2025
[ Upstream commit 5066293 ] KMSAN reported an uninit-value access in vxlan_vnifilter_dump() [1]. If the length of the netlink message payload is less than sizeof(struct tunnel_msg), vxlan_vnifilter_dump() accesses bytes beyond the message. This can lead to uninit-value access. Fix this by returning an error in such situations. [1] BUG: KMSAN: uninit-value in vxlan_vnifilter_dump+0x328/0x920 drivers/net/vxlan/vxlan_vnifilter.c:422 vxlan_vnifilter_dump+0x328/0x920 drivers/net/vxlan/vxlan_vnifilter.c:422 rtnl_dumpit+0xd5/0x2f0 net/core/rtnetlink.c:6786 netlink_dump+0x93e/0x15f0 net/netlink/af_netlink.c:2317 __netlink_dump_start+0x716/0xd60 net/netlink/af_netlink.c:2432 netlink_dump_start include/linux/netlink.h:340 [inline] rtnetlink_dump_start net/core/rtnetlink.c:6815 [inline] rtnetlink_rcv_msg+0x1256/0x14a0 net/core/rtnetlink.c:6882 netlink_rcv_skb+0x467/0x660 net/netlink/af_netlink.c:2542 rtnetlink_rcv+0x35/0x40 net/core/rtnetlink.c:6944 netlink_unicast_kernel net/netlink/af_netlink.c:1321 [inline] netlink_unicast+0xed6/0x1290 net/netlink/af_netlink.c:1347 netlink_sendmsg+0x1092/0x1230 net/netlink/af_netlink.c:1891 sock_sendmsg_nosec net/socket.c:711 [inline] __sock_sendmsg+0x330/0x3d0 net/socket.c:726 ____sys_sendmsg+0x7f4/0xb50 net/socket.c:2583 ___sys_sendmsg+0x271/0x3b0 net/socket.c:2637 __sys_sendmsg net/socket.c:2669 [inline] __do_sys_sendmsg net/socket.c:2674 [inline] __se_sys_sendmsg net/socket.c:2672 [inline] __x64_sys_sendmsg+0x211/0x3e0 net/socket.c:2672 x64_sys_call+0x3878/0x3d90 arch/x86/include/generated/asm/syscalls_64.h:47 do_syscall_x64 arch/x86/entry/common.c:52 [inline] do_syscall_64+0xd9/0x1d0 arch/x86/entry/common.c:83 entry_SYSCALL_64_after_hwframe+0x77/0x7f Uninit was created at: slab_post_alloc_hook mm/slub.c:4110 [inline] slab_alloc_node mm/slub.c:4153 [inline] kmem_cache_alloc_node_noprof+0x800/0xe80 mm/slub.c:4205 kmalloc_reserve+0x13b/0x4b0 net/core/skbuff.c:587 __alloc_skb+0x347/0x7d0 net/core/skbuff.c:678 alloc_skb include/linux/skbuff.h:1323 [inline] netlink_alloc_large_skb+0xa5/0x280 net/netlink/af_netlink.c:1196 netlink_sendmsg+0xac9/0x1230 net/netlink/af_netlink.c:1866 sock_sendmsg_nosec net/socket.c:711 [inline] __sock_sendmsg+0x330/0x3d0 net/socket.c:726 ____sys_sendmsg+0x7f4/0xb50 net/socket.c:2583 ___sys_sendmsg+0x271/0x3b0 net/socket.c:2637 __sys_sendmsg net/socket.c:2669 [inline] __do_sys_sendmsg net/socket.c:2674 [inline] __se_sys_sendmsg net/socket.c:2672 [inline] __x64_sys_sendmsg+0x211/0x3e0 net/socket.c:2672 x64_sys_call+0x3878/0x3d90 arch/x86/include/generated/asm/syscalls_64.h:47 do_syscall_x64 arch/x86/entry/common.c:52 [inline] do_syscall_64+0xd9/0x1d0 arch/x86/entry/common.c:83 entry_SYSCALL_64_after_hwframe+0x77/0x7f CPU: 0 UID: 0 PID: 30991 Comm: syz.4.10630 Not tainted 6.12.0-10694-gc44daa7e3c73 #29 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-3.fc41 04/01/2014 Fixes: f9c4bb0 ("vxlan: vni filtering support on collect metadata device") Reported-by: syzkaller <syzkaller@googlegroups.com> Signed-off-by: Shigeru Yoshida <syoshida@redhat.com> Reviewed-by: Ido Schimmel <idosch@nvidia.com> Link: https://patch.msgid.link/20250123145746.785768-1-syoshida@redhat.com Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The current cargo command creates the default cargo output. this does not look nice in the makefile output
The text was updated successfully, but these errors were encountered: