Use the dummy codegen backend #4648

bjorn3 · 2025-10-23T18:08:47Z

This allows getting rid of all the argument mangling hacks as well as the exported_non_generic_symbols override hack by doing cargo build instead of cargo check. This is also a prerequisite for using native jit mode support in miri that I hope to add to rustc in the future to be used with cg_clif too.

src/bin/miri.rs

cargo-miri/src/phases.rs

bjorn3 · 2025-10-23T18:14:06Z

src/lib.rs

 /// Also disable the MIR pass that inserts an alignment check on every pointer dereference. Miri
 /// does that too, and with a better error message.
 pub const MIRI_DEFAULT_ARGS: &[&str] = &[
+    "-Zcodegen-backend=dummy",


The dummy codegen backend is a builtin codegen backend that is only capable of emitting rlibs containing the crate metadata. For every other crate type it emits an error after codegen. This is fine for miri as for --crate-type bin, miri exits before codegen.

…apkin Allow codegen backends to indicate which crate types they support This way cargo will drop the unsupported crate types for crates that specify multiple crate types. This is a prerequisite for rust-lang/miri#4648.

…, r=WaffleLapkin Allow codegen backends to indicate which crate types they support This way cargo will drop the unsupported crate types for crates that specify multiple crate types. This is a prerequisite for rust-lang/miri#4648.

Rollup merge of #148177 - bjorn3:codegen_backend_crate_types, r=WaffleLapkin Allow codegen backends to indicate which crate types they support This way cargo will drop the unsupported crate types for crates that specify multiple crate types. This is a prerequisite for rust-lang/miri#4648.

cargo-miri/src/phases.rs

…es, r=JonathanBrouwer Handle default features and -Ctarget-features in the dummy backend This prevents a warning about ABI relevant target features not being set on x86 and arm. In addition it is required for miri to report correct features in is_*_feature_detected!() if miri switches to the dummy backend. Required for rust-lang/miri#4648

Rollup merge of #148253 - bjorn3:dummy_backend_target_features, r=JonathanBrouwer Handle default features and -Ctarget-features in the dummy backend This prevents a warning about ABI relevant target features not being set on x86 and arm. In addition it is required for miri to report correct features in is_*_feature_detected!() if miri switches to the dummy backend. Required for rust-lang/miri#4648

bjorn3 · 2025-10-30T15:03:39Z

Almost there. With rust-lang/rust#148177 and rust-lang/rust#148253 merged, the entire miri test suite passes after blessing a single change to a query cycle error. Doctests are broken however due to those requiring a check build for bin crates first, which the dummy codegen backend doesn't yet allow. Opened rust-lang/rust#148299 for this.

bjorn3 · 2025-10-30T16:05:41Z

I hadn't noticed the cargo-miri-test test suite before. It currently has trouble with non-rlib/bin crate types. In particular cargo thinks it needs to build crates only available as cdylib (despite those never being usable) and the dropping unsupported crate type warnings have a non-deterministic order for some reason (edit: I think this is because of non-deterministic order in which rustc is executed. I assumed cargo would skip unsupported --crate-type args and report those warnings itself, but it seems like cargo just forwards them to rustc and expects rustc to produce the warnings.).

Allow check builds with binaries for the dummy codegen backend This is likely the last part necessary for rust-lang/miri#4648. Miri needs to be able to do a regular check build for compile_fail doc tests to work.

RalfJung

If we can get rid of the hacks that patch --emit and --extern, that would definitely neat.

However I am concerned about those "unsupported crate type" warnings, that seems like something Miri users will then also see in the real world. We also need to ensure crate graphs that include cdylib keep working.

View changes since this review

cargo-miri/src/setup.rs

RalfJung · 2025-11-02T10:48:12Z

test-cargo-miri/Cargo.toml

 [dependencies]
 byteorder = "1.0"
-cdylib = { path = "cdylib" }
+#cdylib = { path = "cdylib" }


What's up with this?
We added cdylib dependencies here because we got bug reports about Miri acting strange on them on real code, so we can't just stop supporting that.

Dependencies that are both an rlib and cdylib work just fine. However dependencies that are solely a cdylib won't work. But you wouldn't be able to use them anyway. Cargo for some reason insists on still building them, but you would never be able to use them in any other way than copying them from the target dir. It is not an artifact dependency that you can link against through a build script.

That looks deliberate: 1923044 explicitly talks about "cdylib-only". That commit was part of #1710 which fixed #1705.

#1705 was about a crate that mixed cdylib and other crate types.

But still, a cdylib-only crate might be odd or useless, but it worked fine so far so I am concerned about breaking it as people might be relying on it.

hyper was an rlib+cdylib crate, not a cdylib-only crate. rlib+cdylib still works. Digging through the linked issues, all of them seem to be for rlib+cdylib crates and none for cdylib-only crates.

Yeah that's what I just wrote in the 2nd comment. Hard to say what people rely on though.

What's the error one gets when having a cdylib-only crate?

What's the error one gets when having a cdylib-only crate?

error: cannot produce cdylib for `cdylib v0.1.0 (/home/gh-bjorn3/miri/test-cargo-miri/cdylib)` as the target `aarch64-unknown-linux-gnu` does not support these crate types

I've now moved the cdylib package from a dependency to just a workspace member. This way we still check that the mere presence of a cdylib-only package doesn't break miri, only when using it as dependency.

That error is kind of misleading since it's not about the target, it's about the codegen backend.

Yeah, this is unfortunately hard coded in cargo. I thought it was also hard coded in the code parsing the rustc warnings, but looks like that is not the case after all: https://github.com/rust-lang/cargo/blob/4406c1b96413e79ee770f81506a0697fb8f11404/src/cargo/core/compiler/build_context/target_info.rs#L669-L670 I will open a PR to change the rustc warning and see if I can open a PR to change the cargo error.

RalfJung · 2025-11-02T11:00:26Z

test-cargo-miri/run.custom-target-dir.stderr.ref

@@ -1,2 +1,6 @@
+UNSUPPORTED CRATE TYPE


As I mentioned in the review summary, this isn't great. A lot of crate graphs contain cdylibs, so we'd be showing this warning to users a lot.

RalfJung · 2025-11-02T11:02:15Z

cargo-miri/src/phases.rs

We used to have code here that skips the build if the crate type is cdylib. That was obviously wrong, I just didn't realize at the time that there can be multiple crate types. But we could maybe instead filter the crate types and remove cdylib (and anything else that's unsupported), and then skip the build if there's no crate type left?

I've now pushed a commit to do that.

That should then also fix the cdylib-only-dependency, shouldn't it?

It doesn't. The dummy codegen backend tells cargo that cdylib is not supported and then cargo reports this error before we can filter the --crate-type cdylib.

Ah, unfortunate.

Allow check builds with binaries for the dummy codegen backend This is likely the last part necessary for rust-lang/miri#4648. Miri needs to be able to do a regular check build for compile_fail doc tests to work.

This allows getting rid of all the argument mangling hacks as well as the exported_non_generic_symbols override hack by doing cargo build instead of cargo check. This is also a prerequisite for using native jit mode support in miri that I hope to add to rustc in the future to be used with cg_clif too.

rustbot · 2025-11-03T18:21:05Z

Thank you for contributing to Miri!
Please remember to not force-push to the PR branch except when you need to rebase due to a conflict or when the reviewer asks you for it.

…athanBrouwer Handle default features and -Ctarget-features in the dummy backend This prevents a warning about ABI relevant target features not being set on x86 and arm. In addition it is required for miri to report correct features in is_*_feature_detected!() if miri switches to the dummy backend. Required for rust-lang/miri#4648

RalfJung

@rustbot author

View changes since this review

RalfJung · 2025-11-05T09:47:57Z

src/bin/miri.rs

        if config.opts.prints.is_empty() && self.target_crate {
+            // Avoid warnings about unsupported crate types
+            #[allow(rustc::bad_opt_access)]
+            config.opts.crate_types.retain(|&c| c == CrateType::Executable || c == CrateType::Rlib);


We should probably only do this for target crates?

What happens if the list of crate types is empty after the filtering?

Also, why is it okay to allow bad_opt_access here?

We should probably only do this for target crates?

This is gated behind if self.target_crate.

What happens if the list of crate types is empty after the filtering?

Currently rustc treats that as implicit --crate-type lib, but in any case it shouldn't be possible to get an empty list here. Cargo will error if none of the crate types are bin or rlib as the dummy codegen backend only lists those crate types as supported.

Also, why is it okay to allow bad_opt_access here?

This is long before the TyCtxt gets created that you are normally supposed to access crate_types through.

Currently rustc treats that as implicit --crate-type lib, but in any case it shouldn't be possible to get an empty list here. Cargo will error if none of the crate types are bin or rlib as the dummy codegen backend only lists those crate types as supported.

Please add an assertion then.

Currently rustc treats that as implicit --crate-type lib

Actually, it treats it as --crate-type bin just like if the user didn't pass --crate-type at all. I got it confused with the behavior when --crate-type is present on the cli, but all listed crate types are unsupported by the target or codegen backend.

In any case I added an assertion.

src/bin/miri.rs

test-cargo-miri/Cargo.toml

src/bin/miri.rs

bjorn3 · 2025-11-05T13:17:31Z

@rustbot ready

rustbot added the S-waiting-on-author Status: Waiting for the PR author to address review comments label Oct 23, 2025

bjorn3 commented Oct 23, 2025

View reviewed changes

src/bin/miri.rs Show resolved Hide resolved

bjorn3 commented Oct 23, 2025

View reviewed changes

cargo-miri/src/phases.rs Show resolved Hide resolved

bjorn3 commented Oct 23, 2025

View reviewed changes

bjorn3 mentioned this pull request Oct 27, 2025

Allow codegen backends to indicate which crate types they support rust-lang/rust#148177

Merged

bjorn3 force-pushed the dummy_backend branch 2 times, most recently from ac1e234 to ef6aed4 Compare October 29, 2025 12:23

This comment has been minimized.

Sign in to view

bjorn3 commented Oct 29, 2025

View reviewed changes

cargo-miri/src/phases.rs Outdated Show resolved Hide resolved

bjorn3 mentioned this pull request Oct 29, 2025

Handle default features and -Ctarget-features in the dummy backend rust-lang/rust#148253

Merged

bjorn3 force-pushed the dummy_backend branch 4 times, most recently from a4a79ee to ff9ef3b Compare October 30, 2025 15:00

bjorn3 force-pushed the dummy_backend branch from ff9ef3b to a28a6e5 Compare October 30, 2025 15:09

bjorn3 mentioned this pull request Oct 30, 2025

Allow check builds with binaries for the dummy codegen backend rust-lang/rust#148299

Merged

bjorn3 force-pushed the dummy_backend branch 2 times, most recently from 5f8c2f8 to 38dd4f6 Compare October 30, 2025 15:21

bjorn3 force-pushed the dummy_backend branch 2 times, most recently from 95b1a48 to cad7423 Compare October 31, 2025 12:45

RalfJung reviewed Nov 2, 2025

View reviewed changes

bjorn3 force-pushed the dummy_backend branch from c4b94af to 194fdad Compare November 2, 2025 11:11

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the dummy_backend branch 3 times, most recently from 3233570 to 69bc70b Compare November 3, 2025 16:35

bjorn3 changed the title ~~[WIP] Use the dummy codegen backend~~ Use the dummy codegen backend Nov 3, 2025

bjorn3 force-pushed the dummy_backend branch from 69bc70b to f661d28 Compare November 3, 2025 16:47

Fix test-cargo-miri

df28708

bjorn3 force-pushed the dummy_backend branch from f661d28 to df28708 Compare November 3, 2025 16:48

bjorn3 marked this pull request as ready for review November 3, 2025 18:21

rustbot added S-waiting-on-review Status: Waiting for a review to complete and removed S-waiting-on-author Status: Waiting for the PR author to address review comments labels Nov 3, 2025

RalfJung reviewed Nov 5, 2025

View reviewed changes

rustbot added S-waiting-on-author Status: Waiting for the PR author to address review comments and removed S-waiting-on-review Status: Waiting for a review to complete labels Nov 5, 2025

bjorn3 force-pushed the dummy_backend branch from ee9ffad to ba4b99f Compare November 5, 2025 10:28

Fix review comments

dae6966

bjorn3 force-pushed the dummy_backend branch from ba4b99f to dae6966 Compare November 5, 2025 10:58

RalfJung reviewed Nov 5, 2025

View reviewed changes

src/bin/miri.rs Outdated Show resolved Hide resolved

src/bin/miri.rs Outdated Show resolved Hide resolved

Review comments

79d1d33

rustbot added S-waiting-on-review Status: Waiting for a review to complete and removed S-waiting-on-author Status: Waiting for the PR author to address review comments labels Nov 5, 2025

Use the dummy codegen backend #4648

Are you sure you want to change the base?

Use the dummy codegen backend #4648

Conversation

bjorn3 commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

Uh oh!

bjorn3 commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bjorn3 commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RalfJung left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Nov 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Nov 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

rustbot commented Nov 3, 2025

Uh oh!

RalfJung left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bjorn3 commented Nov 5, 2025

Uh oh!

Reviewers

bjorn3 commented Oct 23, 2025 •

edited

Loading

bjorn3 commented Oct 30, 2025 •

edited

Loading

bjorn3 commented Oct 30, 2025 •

edited

Loading

RalfJung left a comment •

edited by rustbot

Loading

RalfJung Nov 2, 2025 •

edited

Loading

RalfJung Nov 2, 2025 •

edited

Loading

RalfJung left a comment •

edited by rustbot

Loading