bindgen wanted features & bugfixes

[ojeda]

Note: see as well the "rust-for-linux" label in bindgen: https://github.com/rust-lang/rust-bindgen/labels/rust-for-linux.

Features that we would like to see

Required (we almost certainly want them)

• Support for a GCC-based backend.

  • Issue: Support for a GCC-based backend rust-lang/rust-bindgen#1949.

• Support non-trivial object-like macros that evaluate to a numeric value when fully expanded.

  • For instance:
    • An expression, e.g. #define A ((int)1).
    • Macros using function-like macros, e.g. Linux ioctl #defines using _IO*.
  • Issue: Improve macro defined constants to recognize #define CONSTANT ((int) 1) rust-lang/rust-bindgen#316 (expressions).
  • Issue: Functional C macros are not expanded. rust-lang/rust-bindgen#753 (using other macros).
  • PR ("one pass per macro", i.e. option 1 below): Workaround for expansion of function-like macros rust-lang/rust-bindgen#2779 (0.70.0?).
  • PR (follow up fixes): Switch to single precompiled header in macro fallback rust-lang/rust-bindgen#2823 (0.70.0?).
  • Meta-issue: Preprocessing meta-issue rust-lang/rust-bindgen#2426.
  • Zulip: https://rust-for-linux.zulipchat.com/#narrow/stream/288089-General/topic/bindgen.20issue.20.23753.20-.20function.20like.20macros/near/420514477.
  • Cc: @jbaublitz who has been following/helping with this issue since the beginning and potentially interested in working on it.
  • Status:

    I met with @emilio a few days ago (early February 2024), and we discussed this for a bit, including the bindgen-side solution/workaround of emitting extra code so that we can fetch the value.

    The variations we discussed were "one pass per macro" (slow, but could work without user input), "one extra pass for all macros" (this would be fast, but would require the user to pass e.g. a regex of matching macros to evaluate that are "known to be constants" already); as well as the possibility of "appending the macros later on" (i.e. add them into a context or similar, which may be possible or not with the current API; we should also check e.g. what Clang-Repl/Cling does here).

    So, in summary, bindgen would be happy with a workaround. But solving it in LLVM is, of course, the best long-term, so that would be great too (ideally both, so that we can use the feature before projects like the kernel have a minimum LLVM version >= 19).

    For the LLVM side, @AaronBallman (the Clang maintainer) told me he would be happy to review our changes if we sent them, in order to help bindgen, so that is great if someone wants to work on it.

    • The first option, i.e. "one pass per macro" was prototyped by @jbaublitz.

• Support aligned members (including structs with bitfields) inside packed structs.

  • See as well the Rust support for it: Rust wanted features #354.
  • i.e. avoid generating #[repr(C, align(...)] inside an #[repr(C, packed)].
  • Seen for struct alt_instr: https://lore.kernel.org/all/202212272003.rgQDX8DQ-lkp@intel.com/
  • Issue: packed type cannot transitively contain a #[repr(align)] type rust-lang/rust-bindgen#2179.
  • Issue: structure layout using __aligned__ attribute is incorrect rust-lang/rust-bindgen#867.
  • Issue: handle more cases of packed and aligned structs when it's safe/correct rust-lang/rust-bindgen#2725.
  • Related issue: E0587 error on packed and aligned structures from C rust-lang/rust-bindgen#1538.
  • PR: Consider alignment contribution from bitfields rust-lang/rust-bindgen#2388.
  • PR: try to avoid #[repr(packed)] when align is needed rust-lang/rust-bindgen#2734.
  • PR (RFC alternative 1): [RFC, V1] try to exclude packed attr for types that contain aligned types rust-lang/rust-bindgen#2769.
  • PR (RFC alternative 2): [RFC, V2] exclude packed attr for types that contain aligned types when possible rust-lang/rust-bindgen#2770.

• Support a new "enum variant" kind, similar to rustified_enum, that provides a safe method to transform a C enum to Rust.

  • The goal is to ensure at compile-time that, if the C side enum adds a variant, the Rust side (i.e. the conversion method) will be updated properly too.
  • From: https://rust-for-linux.zulipchat.com/#narrow/stream/288089-General/topic/Enum.20handling/near/392620696.
  • Issue: Add enum option constified_enum + rustified_enum with conversions rust-lang/rust-bindgen#2646.
  • PR: Add option to generate safe and unsafe conversions for rustified enums rust-lang/rust-bindgen#2908.
  • Issue (revamping the CLI interface for enums, from Add option to generate safe and unsafe conversions for rustified enums rust-lang/rust-bindgen#2908 (comment)): [RFC] Revamp interface for enum representations rust-lang/rust-bindgen#3050.
  • Issue (modifying existing behavior?): Using rustified enums in function signatures is (almost?) always wrong rust-lang/rust-bindgen#3051.

Nice to have (not critical, we could workaround if needed, etc.)

• Support for Edition 2024.

  • Not important, since we can keep compiling the bindings in an older edition like 2021, but it would be nice to migrate everything to Edition 2024 eventually if it becomes available in Debian Trixie.
  • Edition 2021: Supporting 2021-edition features rust-lang/rust-bindgen#2998.
  • unsafe extern issue: Generate unsafe extern blocks (Edition 2024) rust-lang/rust-bindgen#2901.
  • unsafe extern PR: Add support for unsafe extern blocks rust-lang/rust-bindgen#3015.
  • Issue: Make RustEdition public rust-lang/rust-bindgen#3012.
  • PR: Make RustEdition public rust-lang/rust-bindgen#3013.

• process_comments via CLI.

  • See as well "C javadoc comments" below.
  • Issue: CLI integration with doxygen-rs? rust-lang/rust-bindgen#2953.

Low priority (we will likely not use them in the end)

Done (stabilized, fixed, not needed anymore, etc.)

• Support unsafe_op_in_unsafe_fn.

  • Issue: Support unsafe_op_in_unsafe_fn rust-lang/rust-bindgen#2063.
  • PR: Wrap unsafe function's bodies in unsafe blocks rust-lang/rust-bindgen#2266.

• Support _Noreturn, [[noreturn]], __attribute__((noreturn))

  • Issue: Support _Noreturn, [[noreturn]], __attribute__((noreturn)) rust-lang/rust-bindgen#2094.
  • PR: Handle no return attributes rust-lang/rust-bindgen#2259.

• Support typedefs shadowing types e.g. typedef struct T *T;.

  • In other words, avoid conflating C name spaces.
  • Seen in the past e.g. for maple_*nodes (Build fail on linux-next (next-20220629 tag): the name maple_enode is defined multiple times #795).
  • Issue: Multiple definitions when a pointer gets typedef’d to the same name as a struct rust-lang/rust-bindgen#2227.
  • PR: Handle the const struct * and struct * patterns rust-lang/rust-bindgen#2304.

• Take advantage of feature(core_ffi_c) and feature(c_size_t).

  • This avoids having to manually specify --ctypes-prefix core::ffi.
  • Issue: Take advantage of feature(core_ffi_c) (and possibly feature(c_size_t)) rust-lang/rust-bindgen#2231.
  • PR: use #[feature(core_ffi_c)] when available rust-lang/rust-bindgen#2267.

• A way to query the Clang version being used.

  • Issue: Querying bindgen for LLVM version being used rust-lang/rust-bindgen#2138.
  • PR: options: Implement --version manually and print clang version on --version --verbose rust-lang/rust-bindgen#2140.

• Compile-time layout tests.

  • We are planning to run the layout tests as e.g. KUnit tests, but if they were compile-time ones, it would be simpler.
  • Issue: Feature request: compile-time layout tests rust-lang/rust-bindgen#2786.
  • PR: Compile time layout tests rust-lang/rust-bindgen#2787 (0.xx.x).

• Support static inline functions in headers (by generating a wrapper).

  • Currently we do it manually. bindgen currently (0.64.0) has experimental support via --wrap-static-fns --experimental.
  • Netdev would like it, e.g. skbuff.h has 300+ static inlines: https://lore.kernel.org/rust-for-linux/f28d6403-d042-4ffb-9872-044388d0f9d9@lunn.ch/
  • Tracking (?) Issue: Allow wrap-static-fns to be used without experimental feature rust-lang/rust-bindgen#2900.
  • Issue: Generate C code to export static inline functions rust-lang/rust-bindgen#1090.
  • PR: Generate extern wrappers for inlined functions rust-lang/rust-bindgen#2335.
  • Issue: Interaction between wrapped statics and redefined function rust-lang/rust-bindgen#2420.
  • Stabilization PR: Stabilize --wrap-static-fns rust-lang/rust-bindgen#2928.

• Raw pointer bitfield accessors.

  • Issue: bitfields raw pointer accessors rust-lang/rust-bindgen#2674.
  • PR: Add additional helpers to bitfield data structure rust-lang/rust-bindgen#2876.

• Rust for Linux in the bindgen CI.

  • PR: ci: Add Rust for Linux rust-lang/rust-bindgen#2851.

• CI: update to v6.12-rc5.

  • PR: ci: Move forward Rust for Linux version rust-lang/rust-bindgen#2970.
  • Reported in the CI of PR: fix very small typo rust-lang/rust-bindgen#2969.

• A way to specify a MSRV, instead of --rust-target (which only supports a subset of versions).

  • For instance, when we started using --rust-target, the kernel supported Rust 1.78.0+ and bindgen 0.65.1+, thus we picked --rust-target 1.68, since it is the maximum stable version supported by the --rust-target flag in bindgen 0.65.1 (and it is lower than our MSRV).
  • Lore: https://lore.kernel.org/rust-for-linux/20241123180323.255997-1-ojeda@kernel.org/.
  • Rust Zulip: https://rust-lang.zulipchat.com/#narrow/channel/425075-rust-for-linux/topic/rust.20version.20on.20generated.20bindings/near/484087179.
  • PR: Refactor RustTarget rust-lang/rust-bindgen#2993.
  • PR: Make nightly target compatible with every other target rust-lang/rust-bindgen#3006.

• A way to specify an output Edition, e.g. --rust-edition.

  • PR: Introduce --rust-edition rust-lang/rust-bindgen#3002.
  • PR: Add rust language editions support rust-lang/rust-bindgen#3000.

• CI: update to v6.13-rc1.

  • PR: ci: Move forward Rust for Linux version to v6.13-rc1 rust-lang/rust-bindgen#3030.

---

Bugs that we would like to see fixed

Required (we almost certainly want them)

• BindgenUnion can get incorrect alignment on i686.
  • Issue: BindgenUnion can get incorrect alignment on i686 rust-lang/rust-bindgen#2773.

Nice to have (probably not critical, we could workaround if needed, etc.)

Low priority (we will likely not use them in the end)

Done (stabilized, fixed, or not needed anymore, etc.)

• Support --version without header (affects 0.69.0, fixed in 0.69.1, regressed in 0.71.0, fixed again in 0.71.1).

  • We currently workaround it with a dummy parameter.
  • Commit: 9e98db1 ("rust: work around bindgen 0.69.0 issue").
  • Issue (0.69.0): --version no longer works without <HEADER> argument rust-lang/rust-bindgen#2677.
  • Issue (0.71.0): bindgen --version does not work (0.71.0 regression) rust-lang/rust-bindgen#3039.
  • Issue (probably related, 0.71.0): generate shell completion is changed in v0.71.0 rust-lang/rust-bindgen#3037.
  • PR: fix(args): make header argument optional rust-lang/rust-bindgen#2678 (0.69.1).
  • PR: Unbreak shell completion and --version without header rust-lang/rust-bindgen#3040 (0.71.1).

• C javadoc comments are not Markdown-escaped, triggering rustdoc warnings.

  • Issue: C javadoc comments are not Markdown-escaped, triggering rustdoc warnings rust-lang/rust-bindgen#2057.

• Attribute btf_type_tag makes constness of pointers disappear.

  • It will allow to undo the f6bb7cb workaround.
  • Should be fixed in 0.61.
  • Issue: Attribute btf_type_tag makes constness of pointers disappear  rust-lang/rust-bindgen#2244.
  • PR: ty: Use canonical type to access pointee. rust-lang/rust-bindgen#2245.

• "Unable to find libclang" problems (rust compiler not found #934), possibly related to Clang 15.

  • Issue: Unable to find libclang: "the libclang shared library rust-lang/rust-bindgen#2360.
  • PR (may or may not solve our issue): Bump clang-sys version rust-lang/rust-bindgen#2368.

• --wrap-static-fns generates code that returns a call to a void function.

  • Issue: --wrap-static-fns generates code that returns a call to a void function rust-lang/rust-bindgen#2485.
  • PR: Do not return when calling void functions  rust-lang/rust-bindgen#2497.

• --wrap-static-fns generates () for empty parameter lists.

  • Issue: --wrap-static-fns generates () for empty parameter lists rust-lang/rust-bindgen#2484.
  • Should be fixed in 0.65.1.

• Fix the support for linking libclang statically.

  • Zulip: https://rust-for-linux.zulipchat.com/#narrow/stream/291565-Help/topic/Build.20errors/near/391197677.
  • PR: Fix bindgen-cli support link static libclang rust-lang/rust-bindgen#2644.

[pvdrz]

Hi everyone!

I have opened some PRs on bindgen adding some features you needed:

• Handle no return attributes rust-lang/rust-bindgen#2259
• Wrap unsafe function's bodies in unsafe blocks rust-lang/rust-bindgen#2266
• use #[feature(core_ffi_c)] when available rust-lang/rust-bindgen#2267

I'd be happy to hear if the PRs here actually provide what you need.

[alex]

These look spot on to me. Thanks for your efforts!

[ojeda]

Added links above, thanks a lot @pvdrz! :)

[dwlsalmeida]

@ojeda Hey Miguel, is there a workaround in place for this?

Support function-like macros that use other macros and that evaluate to a numeric value when fully expanded (e.g. for Linux ioctl #defines using _IO*).

I am trying to write a v4l2 driver and one of the issues I found was that the pixel formats bindings are not generated by bindgen because the V4L2_PIX_FMT_* stuff is defined using the v4l2_fourcc macro

[ojeda]

For something like pixel formats that are stable (given they are in uapi, right?), I guess the easiest is re-implementing them on the Rust side, especially if you want to give them a proper type etc. anyway. Probably using a small temporary script to perform the bulk of the conversion.

Otherwise, if performance is not too much of a concern (even if cross-language LTO could reduce the cost at some point), one alternative is "executing" the macro at runtime by providing a C function that does it for the Rust side. In your case this one is likely not interesting because the important bits are the parameters to the macro, rather than the macro itself.

[bjorn3]

For the drm fourcc pixel format codes, the drm-fourcc crate has some code which fetches all #define from the header, generates a constant for each #define in a new header and then runs bindgen on the generated header: https://github.com/danielzfranklin/drm-fourcc-rs/blob/main/build.rs

[dwlsalmeida]

One question though, suppose I use the approach suggested by @bjorn3

Do I commit the bindings I generate? Asking because apparently bindings_generated.rs is not part of the kernel per-se and is only generated during build time.

[bjorn3]

I think the bindings should be generated on the fly to ensure they stay up to date, just like bindings_generated.rs.

[dwlsalmeida]

well, in which case I suppose that the code to generate them (i.e. something similar to the code in the drm-fourcc crate) would have to be integrated into the kernel build system somehow?

[bjorn3]

Indeed. There is already infrastructure to run code written in rust. See for example scripts/generate_rust_target.rs.

[dwlsalmeida]

Makes sense, thanks for the pointers!

[ojeda]

It depends. If these constants are strictly stable (unlike many kernel APIs that you may see in bindings_generated.rs), you have the possibility to commit them, but in that case I would recommend writing directly in Rust what you need (rather than raw bindings that then you wrap). This is why I mentioned above the stability. You could also have a test to verify both sides match, though that could be a one-off thing.

If you go the generated route (which many times is the only way), then indeed, you could use a host program (it does not necessarily need to be written in Rust, but as @bjorn3 mentions, the support is there, so I would recommend it -- though currently you will only have the Rust standard library available, e.g. no regex). This approach has the drawback of adding complexity (the host program) and, depending on how you implement/parse the C header, you may introduce constraints on the C side. Another similar option is a proc macro, but that is slower (triggered every time you recompile the kernel crate, which would be unneeded -- you only need to do it once) and non-"pure" (in the "do not use the environment from a proc macro" sense).

Another angle to think about is whether new constants will appear frequently or not.