Miri read_discriminant: return a scalar instead of raw underlying bytes #72419

RalfJung · 2020-05-21T08:21:08Z

RalfJung · 2020-05-21T08:29:57Z

src/librustc_mir/interpret/operand.rs

-                }
+                };
+                // Compute the size of the scalar we need to return.
+                // FIXME: Why do we not need to do a cast here like we do above?


This one particularly confused me -- we should resolve that question before landing.

For the niche layout, the variant indices have to not overflow host u32 nor the target (type-checking) discriminant type (isize by default but I guess it could be u8/i8).

So you can add an assert that it fits as a positive number in the discriminant type, but no cast should be necessary.

Scalar::from_u32 already has such an assert (at least on debug builds).

RalfJung · 2020-05-22T09:37:38Z

This makes the generator-related Miri tests fail:

thread 'rustc' panicked at 'assertion failed: `(left == right)`
  left: `Size { raw: 4 }`,
 right: `Size { raw: 8 }`: Size mismatch when writing bits', /home/r/src/rust/rustc.2/src/libstd/macros.rs:16:9

I'll need to investigate.

src/librustc_mir/interpret/operand.rs

eddyb · 2020-05-22T17:39:41Z

src/librustc_mir/interpret/operand.rs

+        // - The field storing the discriminant has a layout, which my be a pointer.
+        //   This is `discr_val.layout`; we just use it for sanity checks.


"storing the discriminant" could be something more like "encoding the discriminant as a tag/niche".

That might make it clearer why the type differs: the only reason to have a simple encoding is cheap decoding, it could be any arbitrary mapping as long as you have enough values for every possible discriminant to be represented.

"encoding the discriminant as a tag/niche".

But the layout of the tag/niche is discr_layout, right? The second item in my list? And then discr_ty is the high-level thing?

But the layout of the tag/niche is discr_layout, right?

If this is correct I'll rename the variables to tag_layout or so.

Did that, and also opened #72497 for the general terminology cleanup.

eddyb · 2020-05-22T17:40:38Z

src/librustc_mir/interpret/operand.rs

+        // - The discriminant has a layout for tag storing purposes, which is always an integer.
+        //   This is `discr_layout` and is used to interpret the value we read from the
+        //   discriminant field.


Not sure what this refers to. Is this a tag/niche-sized integer? So for e.g. a pointer niche, it would be a pointer-sized integer?

This is the layout stored in Variants::Multiple::discr. So I think the answer to your questions is "yes".

src/librustc_mir/interpret/operand.rs

eddyb

LGTM modulo comments (@oli-obk could take a look at the more miri-specific parts of this, which I'm not too familiar with)

src/librustc_mir/interpret/operand.rs

RalfJung · 2020-05-23T15:27:50Z

This PR now also includes a discriminant_ty helper method to determine the discriminant type for any type. This is shared between Rvalue type computation and Miri. And indeed it fixes a bug in the former which uses isize as discriminant type for non-enum ADTs (instead of u8).

RalfJung · 2020-05-25T13:34:43Z

Rebased as my cast-cleanup-PR also included a commit from this one.

oli-obk · 2020-05-26T18:12:52Z

@bors r=oli-obk,eddyb

bors · 2020-05-26T18:12:54Z

📌 Commit 95b853c has been approved by oli-obk,eddyb

@oli-obk

…bk,eddyb Miri read_discriminant: return a scalar instead of raw underlying bytes r? @oli-obk @eddyb

RalfJung · 2020-05-29T19:56:41Z

Failed in #72737 (comment)
@bors r-

RalfJung · 2020-05-29T22:03:05Z

Fixes type sanity check assertions.
@bors r=oli-obk,eddyb

bors · 2020-05-29T22:03:05Z

📌 Commit c4b6224 has been approved by oli-obk,eddyb

@ghost

Rollup of 10 pull requests Successful merges: - rust-lang#72033 (Update RELEASES.md for 1.44.0) - rust-lang#72162 (Add Extend::{extend_one,extend_reserve}) - rust-lang#72419 (Miri read_discriminant: return a scalar instead of raw underlying bytes) - rust-lang#72621 (Don't bail out of trait selection when predicate references an error) - rust-lang#72677 (Fix diagnostics for `@ ..` binding pattern in tuples and tuple structs) - rust-lang#72710 (Add test to make sure -Wunused-crate-dependencies works with tests) - rust-lang#72724 (Revert recursive `TokenKind::Interpolated` expansion for now) - rust-lang#72741 (Remove unused mut from long-linker-command-lines test) - rust-lang#72750 (Remove remaining calls to `as_local_node_id`) - rust-lang#72752 (remove mk_bool) Failed merges: r? @ghost

@eddyb

tag/niche terminology cleanup The term "discriminant" was used in two ways throughout the compiler: * every enum variant has a corresponding discriminant, that can be given explicitly with `Variant = N`. * that discriminant is then encoded in memory to store which variant is active -- but this encoded form of the discriminant was also often called "discriminant", even though it is conceptually quite different (e.g., it can be smaller in size, or even use niche-filling). After discussion with @eddyb, this renames the second term to "tag". The way the tag is encoded can be either `TagEncoding::Direct` (formerly `DiscriminantKind::Tag`) or `TagEncoding::Niche` (formerly `DiscrimianntKind::Niche`). This finally resolves some long-standing confusion I had about the handling of variant indices and discriminants, which surfaced in rust-lang#72419. (There is also a `DiscriminantKind` type in libcore, it remains unaffected. I think this corresponds to the discriminant, not the tag, so that seems all right.) r? @eddyb

@eddyb

tag/niche terminology cleanup The term "discriminant" was used in two ways throughout the compiler: * every enum variant has a corresponding discriminant, that can be given explicitly with `Variant = N`. * that discriminant is then encoded in memory to store which variant is active -- but this encoded form of the discriminant was also often called "discriminant", even though it is conceptually quite different (e.g., it can be smaller in size, or even use niche-filling). After discussion with @eddyb, this renames the second term to "tag". The way the tag is encoded can be either `TagEncoding::Direct` (formerly `DiscriminantKind::Tag`) or `TagEncoding::Niche` (formerly `DiscrimianntKind::Niche`). This finally resolves some long-standing confusion I had about the handling of variant indices and discriminants, which surfaced in rust-lang#72419. (There is also a `DiscriminantKind` type in libcore, it remains unaffected. I think this corresponds to the discriminant, not the tag, so that seems all right.) r? @eddyb

@eddyb

tag/niche terminology cleanup The term "discriminant" was used in two ways throughout the compiler: * every enum variant has a corresponding discriminant, that can be given explicitly with `Variant = N`. * that discriminant is then encoded in memory to store which variant is active -- but this encoded form of the discriminant was also often called "discriminant", even though it is conceptually quite different (e.g., it can be smaller in size, or even use niche-filling). After discussion with @eddyb, this renames the second term to "tag". The way the tag is encoded can be either `TagEncoding::Direct` (formerly `DiscriminantKind::Tag`) or `TagEncoding::Niche` (formerly `DiscrimianntKind::Niche`). This finally resolves some long-standing confusion I had about the handling of variant indices and discriminants, which surfaced in rust-lang#72419. (There is also a `DiscriminantKind` type in libcore, it remains unaffected. I think this corresponds to the discriminant, not the tag, so that seems all right.) r? @eddyb

@eddyb

tag/niche terminology cleanup The term "discriminant" was used in two ways throughout the compiler: * every enum variant has a corresponding discriminant, that can be given explicitly with `Variant = N`. * that discriminant is then encoded in memory to store which variant is active -- but this encoded form of the discriminant was also often called "discriminant", even though it is conceptually quite different (e.g., it can be smaller in size, or even use niche-filling). After discussion with @eddyb, this renames the second term to "tag". The way the tag is encoded can be either `TagEncoding::Direct` (formerly `DiscriminantKind::Tag`) or `TagEncoding::Niche` (formerly `DiscrimianntKind::Niche`). This finally resolves some long-standing confusion I had about the handling of variant indices and discriminants, which surfaced in rust-lang#72419. (There is also a `DiscriminantKind` type in libcore, it remains unaffected. I think this corresponds to the discriminant, not the tag, so that seems all right.) r? @eddyb

rust-highfive assigned oli-obk May 21, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 21, 2020

RalfJung commented May 21, 2020

View reviewed changes

RalfJung force-pushed the read-discriminant branch from 3d65f0c to daceb18 Compare May 21, 2020 08:41

RalfJung mentioned this pull request May 21, 2020

fix discriminant_value sign extension #72425

Merged

This comment has been minimized.

Sign in to view

RalfJung force-pushed the read-discriminant branch from daceb18 to 824fa23 Compare May 22, 2020 07:17

eddyb reviewed May 22, 2020

View reviewed changes

src/librustc_mir/interpret/operand.rs Outdated Show resolved Hide resolved

eddyb reviewed May 22, 2020

View reviewed changes

src/librustc_mir/interpret/operand.rs Outdated Show resolved Hide resolved

eddyb approved these changes May 22, 2020

View reviewed changes

RalfJung force-pushed the read-discriminant branch from 20f6874 to 0af3ac0 Compare May 23, 2020 10:26

RalfJung mentioned this pull request May 23, 2020

tag/niche terminology cleanup #72497

Merged

This comment has been minimized.

Sign in to view

RalfJung commented May 23, 2020

View reviewed changes

src/librustc_mir/interpret/operand.rs Outdated Show resolved Hide resolved

RalfJung force-pushed the read-discriminant branch from 0af3ac0 to 7220175 Compare May 23, 2020 12:52

This comment has been minimized.

Sign in to view

RalfJung force-pushed the read-discriminant branch 3 times, most recently from 27bb80b to 9fbdad0 Compare May 25, 2020 13:33

RalfJung added 6 commits May 25, 2020 15:42

Miri: refactor read_discriminant and make it return Scalar

0f18203

comments and refactor variable names

5a3971c

assert that types without discriminant use variant idx of 0

f8f8033

Format and more tracing

d94923e

Add helper method for determining the type of a discriminant

64fbe2f

fix discriminant_ty for non-enums

ad7179d

RalfJung force-pushed the read-discriminant branch from 9fbdad0 to 95b853c Compare May 25, 2020 13:45

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 26, 2020

RalfJung added a commit to RalfJung/rust that referenced this pull request May 29, 2020

Rollup merge of rust-lang#72419 - RalfJung:read-discriminant, r=oli-o…

6d7d00b

…bk,eddyb Miri read_discriminant: return a scalar instead of raw underlying bytes r? @oli-obk @eddyb

RalfJung mentioned this pull request May 29, 2020

Rollup of 8 pull requests #72737

Closed

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 29, 2020

more type sanity checks in Miri

c4b6224

RalfJung force-pushed the read-discriminant branch from 95b853c to c4b6224 Compare May 29, 2020 22:02

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 29, 2020

JohnTitor mentioned this pull request May 30, 2020

Rollup of 10 pull requests #72768

Merged

bors merged commit a5fb7fc into rust-lang:master May 30, 2020

RalfJung deleted the read-discriminant branch May 30, 2020 12:06

RalfJung mentioned this pull request Jun 5, 2020

ICE: index out of bounds: the len is 0 but the index is 0' with --emit mir and mir-opt-level >1 #72181

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Miri read_discriminant: return a scalar instead of raw underlying bytes #72419

Miri read_discriminant: return a scalar instead of raw underlying bytes #72419

RalfJung commented May 21, 2020

RalfJung May 21, 2020

eddyb May 22, 2020

RalfJung May 23, 2020

This comment has been minimized.

RalfJung commented May 22, 2020

eddyb May 22, 2020

RalfJung May 23, 2020

RalfJung May 23, 2020

RalfJung May 23, 2020

eddyb May 22, 2020

RalfJung May 23, 2020

eddyb left a comment

This comment has been minimized.

This comment has been minimized.

RalfJung commented May 23, 2020

RalfJung commented May 25, 2020

oli-obk commented May 26, 2020

bors commented May 26, 2020

RalfJung commented May 29, 2020

RalfJung commented May 29, 2020

bors commented May 29, 2020

		// - The field storing the discriminant has a layout, which my be a pointer.
		// This is `discr_val.layout`; we just use it for sanity checks.

Miri read_discriminant: return a scalar instead of raw underlying bytes #72419

Miri read_discriminant: return a scalar instead of raw underlying bytes #72419

Conversation

RalfJung commented May 21, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment has been minimized.

RalfJung commented May 22, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eddyb left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

RalfJung commented May 23, 2020

RalfJung commented May 25, 2020

oli-obk commented May 26, 2020

bors commented May 26, 2020

RalfJung commented May 29, 2020

RalfJung commented May 29, 2020

bors commented May 29, 2020