-
Notifications
You must be signed in to change notification settings - Fork 14.1k
Explicitly export core and std macros #139493
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Explicitly export core and std macros #139493
Conversation
|
r? @ChrisDenton rustbot has assigned @ChrisDenton. Use |
|
r? @Amanieu |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@Amanieu the tidy issue highlights an annoying and unforeseen side-effect of this change. The fn xx(i: vec::IntoIter<i32>) {
let _ = i.as_slice();
}
fn main() {}that currently doesn't compile on stable would now compile. Initially I thought this would cause name collisions if users define their own |
|
There's an issue for this change - #53977. |
|
@Voultapher, avoiding the vec module re-export can be done like this: #[macro_export]
macro_rules! myvec {
() => {};
}
pub mod myvec {
pub struct Vec;
}
pub mod prelude {
// Bad: re-exports both macro and type namespace
// pub use crate::myvec;
mod vec_macro_only {
#[allow(hidden_glob_reexports)]
mod myvec {}
pub use crate::*;
}
pub use self::vec_macro_only::myvec;
}
fn main() {
prelude::myvec!();
let _: prelude::myvec::Vec; // error
} |
|
|
This comment has been minimized.
This comment has been minimized.
|
@Voultapher Based on the CI failure I think that a try build would fail now. |
|
Ok, I'll try to get the CI passing first. |
|
@petrochenkov I went through all macros and searched the docs and |
This comment has been minimized.
This comment has been minimized.
|
@Amanieu this program previously worked: use std::*;
fn main() {
panic!("panic works")
}and now runs into: I don't see how we can resolve that without changing language import rules and or special casing the prelude import. |
|
@petrochenkov Do you have any ideas about that? |
|
Could you add a test making sure that the modules |
The ambiguity wouldn't happen if it was the same panic in std root and in the stdlib prelude. Previously |
|
The question is, "what is the exact mechanism and set of rules that cause this to be accepted -- which resolution is preferred and under what circumstances -- and are these being newly added or exposed to stable by this PR?". That is, if we're adding a rule to name resolution such as, "if Above, in an example, you suggest that the resolution in this case will be in favor of
In testing, for Rust 2018, I'm seeing this expansion: #![feature(prelude_import)]
#![no_std]
extern crate core;
#[prelude_import]
use core::prelude::rust_2018::*;
extern crate std;
use std::prelude::v1::*;
fn xx() {
// resolves to core::panic
{ ::std::rt::begin_panic("explicit panic") };
}(And similarly in Rust 2015.) What's happening here? |
|
I've updated the stabilization report.. |
|
☔ The latest upstream changes (presumably #148434) made this pull request unmergeable. Please resolve the merge conflicts. |
|
@traviscross can you explain how you got that expansion? I tried reproducing your results on play.rust-lang.org and got a result consistent with what @Voultapher described: Notably, your expansion lacks As for explaining things in the reference and what the hack is doing, I think I'm at the point where I have a good explanation though I want to go ahead and pull @Voultapher's code and test some expectations to make sure I'm correct on everything. The main question I'm digging into right now is "Is it true that we didn't previously have panic resolution ambiguity because the macrouseprelude is higher priority than stdlibprelude?" My gut says no. Unless I'm missing something we don't currently export the panic macro from the stdlibprelude at all, only the macrouseprelude, so prior to this change when resolving the panic macro in the example given we'd only ever get one candidate, the one from the macrouseprelude. Even if there were two candidates, I think it would behave as described but not for the reason given. It's not that "one is higher priority than the other". Its true that the macrouseprelude is visited first and is treated as the innermost candidate but if they both have a candidate what matters is whether or not shadowing is permitted. I'm pretty sure it would be permitted but only because one of the resolutions would be via a glob import, I thiiink the macrouseprelude resolution would count as an "item in the same scope" for the purposes of https://doc.rust-lang.org/reference/items/use-declarations.html#r-items.use.glob.shadowing. This change moving macros out of the macrouseprelude and into the various stdlibpreludes is what causes the ambiguities. In the problematic example we're going from:
to
which then immediately runs afoul of the As I understand things the only thing the "hack" in this PR is doing is downgrading the ambiguity error to a warning specifically when both candidates are from the builtin panic macros. One thing I want to test to verify this is if I can trigger the warning downgrade logic independently of the preludes. I expect that by creating two new modules, one of which imports std::panic, and the other importing core::panic, if I glob import both and call panic it should also downgrade that ambiguity error to a warning (I also want to see if it matters if I re-export the macros themselves under different names or if I define custom macros with the same name, my guess is that it won't matter and everything will be well behaved as implemented). |
That's correct. The expansion was produced by the code in this PR. |
|
Thanks @yaahc for investigating here. One observation for your consideration on that: The stabilization report says:
However, this program is rejected on nightly but accepted (with a warning) on this branch: use ::core::*;
fn f() {
panic!();
}Similarly, this is rejected on nightly but accepted (with a warning) under this branch: #![no_std]
extern crate std;
use ::std::*;
fn f() {
panic!();
} |
|
This code goes from failing on nightly to passing on this PR in Rust 2018: #![no_std]
extern crate std;
use ::std::prelude::v1::*;
fn f() {
panic!(std::string::String::new());
} |
|
Good catch, indeed there are programs that are now accepted that weren't previously, I got that part wrong. I'll update the stabilization report to reflect this, but I'm unsure how best to do it. |
| == extern_prelude_flag_binding | ||
| && extern_prelude_item_binding.is_some() | ||
| && extern_prelude_item_binding != Some(innermost_binding); | ||
| let is_issue_145575_hack = || { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was this changed to a closure to avoid evaluating the expression if kind is None?
| // These macros needs special handling, so that we don't export it *and* the modules of the same | ||
| // name. We only want the macro in the prelude. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clarification because I was confused at first about what this was doing until I read the comment thread:
| // These macros needs special handling, so that we don't export it *and* the modules of the same | |
| // name. We only want the macro in the prelude. | |
| // These macros needs special handling, so that we don't export it *and* the modules of the same | |
| // name. We only want the macros in the prelude so we shadow the original modules with private | |
| // modules with the same names. |
building on this: So it turns check of orig_ident.name matches sym::panic does depend specifically upon the name being looked up, not the name of the underlying resolution candidates. That means with this change This produces an error: #![no_std]
extern crate std;
mod m1 {
pub use std::prelude::v1::panic as p;
}
mod m2 {
pub use core::prelude::v1::panic as p;
}
fn xx() {
use m1::*;
use m2::*;
p!();
}And this triggers the future compat warning downgrade because both macros are considered builtins and they're being resolved through the #![no_std]
extern crate std;
mod m1 {
pub use std::prelude::v1::env as panic;
}
use m1::*;
fn xx() {
panic!();
} |
which candidate is being selected when we downgrade an ambiguity to a warning?The answer is "the first candidate visited" which will always be the explicit glob import, never the prelude import (assuming there is a prelude import, I'm going to explore how So in addition to the example @traviscross gave where , where the code fails on nightly but passes on 2018 w/ this PR. The inverse example w/ the prelude import coming from //@ edition: 2018
use ::core::prelude::v1::*;
#[allow(unused)]
fn f() {
panic!(std::string::String::new());
}passes on nightly because name resolution selects |
How does
|
|
Based on the edit in the previous comment, I wanted to see if I could trigger the mod m1 {
pub use core::prelude::v1::*;
}
mod m2 {
pub use std::prelude::v1::*;
}
fn foo() {
use m1::*;
use m2::*;
panic!();
}This code is currently accepted and produces an error w/ this change. Prior to this change it would not find a panic candidate in either glob import, so it would have a single unambiguous panic candidate and compile with no issue. After this change it first resolves both the Notably, if you change the order of the glob imports it no longer produces the globvsouter downgraded warning. Instead both candidates end up resolving to |
Here's my attempt at rewriting the associated bulletpoint
This lint is tied to a new exception to the name resolution logic in compiler/rustc_resolve/src/ident.rs similar to an exception added for #145575. This change introduces a new ambiguity warning downgrade path when encountering ambiguous resolution candidates for a This change downgrades that ambiguity error to a warning when both candidates are builtin macros and they're being resolved via the ident #![no_std]
extern crate std;
use std::prelude::v1::*;
fn xx() {
panic!(); // resolves to std::panic
//~^ WARNING `panic` is ambiguous
//~| WARNING this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
}This lint can also trigger in some cases that were previously an error allowing them to compile with a warning: #139493 (comment). In all cases where this lint triggers, the code that is allowed to compile will have a future incompatibility warning associated with it, regardless of whether the code was previously allowed to compile or not. This lint covers all of the common ambiguity errors that crater caught being introduced by this change. It is possible to produce other ambiguity errors not covered by this lint exception that were previously unambiguous prior to this change: #139493 (comment). This issue has not been observed in the wild. |
|
One last thing I'm still trying to understand is why we can't just select I know this ties into the issue identified here: #65721 (comment) but I don't fully understand the issue or how it affects this PR. My current conclusion is that I think in this specific situation we may be able to get away with it because we're actually dealing with the inverse situation from the one vadim highlighted in the above link. Instead of wanting to tiebreak in favor of the manual glob import we want to always tiebreak to the prelude when downgrading ambiguity. My understanding isthat in the other direction we have to prove that we never speculatively resolve to the prelude before materializing the candidate from the manual glob import and then finalize the resolution as from the glob import, but in this change's scenario we know that the panic macro from the prelude will always have been materialized, and we only need to insure we don't greedily speculatively resolve to the glob import when it has also been materialized and catch this specific ambiguity case pre-finalization. I have a WIP PR demonstrating this that I'll push shortly Edit: https://github.com/yaahc/rust/pull/5/commits Here's a PR showing the two commits I setup, the first one adds test cases for all the examples we've recently discussed in this PR and shows how they currently behave with this PR as written. The second commit shows some additions to this PR I think may be warranted and how they impact the test results. I've not gone and explicitly documented how these tests behave on nightly rust, this is left as an exercise for the reader, bonus points if you go back and leave comments on the test cases w/ the info (I gotta go home). I did add one additional testcase, I realized that its possible to trigger the ambiguity warning downgrade w/o involving the prelude so long as the different user glob imports are not in the same scope, in which case they trigger the glob vs outer logic and the associated warning downgrade. |
|
Thanks @yaahc for the detailed analysis. In particular, this answers a lot of questions for me: mod m {
pub use core::env as panic;
}
use m::*;
fn f() {
panic!("PATH");
//[nightly]~^ error: `panic` is ambiguous
//[this-pr]~^ warning: `panic` is ambiguous
}It'd be good to include this in the Reference along with the rule for it. |
|
On the #![no_implicit_prelude]
fn f() {
panic!();
//[nightly]~^ error: cannot find macro `panic` in this scope
//[this-pr]~^ OK
}Comparatively, of course: #![no_implicit_prelude]
fn f() {
drop(()); //~ error: cannot find function `drop` in this scope
}Since the goal of this PR is to move macros to path-based resolution, I'd expect them, if possible, to work in the same way as other names resolved via path-based resolution with respect to |
|
In talking with @yaahc, she described the following (undocumented and maybe surprising) truths about the behavior today: The #![no_implicit_prelude]
fn f() {
assert!(true);
}Is this: #![feature(prelude_import)]
#![no_implicit_prelude]
#[macro_use]
extern crate std;
#[prelude_import]
use std::prelude::rust_2024::*;
fn f() { if !true { ::core::panicking::panic("assertion failed: true") }; }(Some macros are exported by path from the prelude today, e.g. The This behavior was introduced in #62086, and specifically: Speculating, this may have been done because, prior to that PR, certain macros such as So as to not break code, what the PR did was add these macros to the standard library prelude and add a hack that unconditionally visits the standard library prelude, even when As applied here, then, the unfortunate part would be extending the surface area of this hack to all macros, not just the built-in ones. What maybe we could do, then, to avoid this, is to make this hack more specific. We could check whether a macro is one of the ones that is currently (today) exported from the prelude when deciding whether to resolve the name from the standard library prelude. |
#![no_implicit_prelude]
fn f() {
panic!();
//[nightly]~^ error: cannot find macro `panic` in this scope
//[this-pr]~^ OK
}@traviscross are you sure this is the behavior? I don't have access to my dev machine right now and can't check it but it seems to be the opposite of my previous testing. As noted in the stabilization, it interacts with #![no_implicit_prelude]
// Uncomment to fix error.
// use std::{panic, vec};
fn main() {
let _ = vec![3, 6];
panic!(); // Ok on nighlty but error with this PR.
} |
Retested. Yes, this is the behavior I see in testing. For this code: #![no_implicit_prelude]
fn f() {
panic!();
//[nightly]~^ error: cannot find macro `panic` in this scope
//[this-pr]~^ OK
}Run with: rustc +stage1 --edition=2024 --crate-type=rlib src/lib.rsIt compiles without error on my build of the branch of this PR. On nightly, it produces this error: |
ed65c75 to
af5665b
Compare
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
This comment has been minimized.
This comment has been minimized.
|
I pushed up a test demonstrating the behavior. |
This demonstrates that, under the behavior introduced in PR 139493, that the `panic` macro is resolved even with `no_implicit_prelude` and without the macro being explicitly imported. This happens due to a hack introduced in PR 62086 that causes macros in the standard library prelude to be resolved even when `no_implicit_prelude` is present. The fact that PR 139493 adds `panic` to the standard library prelude then triggers this behavior. In Rust 2015, this code was already accepted for a different reason (due to resolution from the macro prelude not being suppressed with `no_implicit_prelude`.) It may not be desirable to extend the scope of this hack to macros for which it did not already have effect.
f3cd775 to
82371a1
Compare
|
Thanks for the test. Strange ... very strange I'll look into it. |
Currently all core and std macros are automatically added to the prelude via #[macro_use]. However a situation arose where we want to add a new macro
assert_matchesbut don't want to pull it into the standard prelude for compatibility reasons. By explicitly exporting the macros found in the core and std crates we get to decide on a per macro basis and can later add them via the rust_20xx preludes.Closes #53977
Unlocks #137487
Reference PR:
Stabilization report lib
Everything N/A or already covered by lang report except, breaking changes: The unstable and never intended for public use
format_args_nlmacro is no longer publicly accessible as requested by @petrochenkov. Affects <10 crates including dependencies.Stabilization report lang
Summary
Explicitly export core and std macros.
This change if merged would change the code injected into user crates to no longer include #[macro_use] on extern crate core and extern crate std. This change is motivated by a near term goal and a longer term goal. The near term goal is to allow a macro to be defined at the std or core crate root but not have it be part of the implicit prelude. Such macros can then be separately promoted to the prelude in a new edition. Specifically this is blocking the stabilization of assert_matches #137487. The longer term goal is to gradually deprecate #[macro_use]. By no longer requiring it for standard library usage, this serves as a step towards that goal. For more information see #53977.
PR link: #139493
Tracking:
ambiguous_panic_imports#147319Reference PRs:
cc @rust-lang/lang @rust-lang/lang-advisors
What is stabilized
Stabilization:
#[macro_use]is no longer automatically included in the crate root module. This allows the explicit import of macros in thecoreandstdprelude e.g.pub use crate::dbg;. No code that was previously not accepted will be accepted now.ambiguous_panic_importslint. No code that was previously not accepted will be accepted now. Code that previously passed without warnings, but included the following or equivalent - only pertaining to core vs std panic - will now receive a warning:This lint is tied to a new exception to the name resolution logic in compiler/rustc_resolve/src/ident.rs similar to an exception added for Glob import causes ambiguity on nightly #145575. Specifically this only happens if the import of two builtin macros is ambiguous and they are named
sym::panic. I.e. this can only happen forcore::panicandstd::panic. While there are some tiny differences in what syntax is allowed instd::panicvscore::panicin editions 2015 and 2018, see. The behavior at runtime will always be the same if it compiles, implying minimal risk in what specific macro is resolved. At worst some closed source project not captured by crater will stop compiling because a different panic is resolved than previously and they were using obscure syntax likepanic!(&String::new()).Design
N/A
Reference
RFC history
N/A
Answers to unresolved questions
N/A
Post-RFC changes
N/A
Key points
Nightly extensions
N/A
Doors closed
No known doors are closed.
Feedback
Call for testing
No.
Nightly use
N/A
Implementation
Major parts
The key change is compiler/rustc_builtin_macros/src/standard_library_imports.rs removing the macro_use inject and the
v1.rspreludes now explicitlypub useing the macros https://github.com/rust-lang/rust/pull/139493/files#diff-a6f9f476d41575b19b399c6d236197355556958218fd035549db6d584dbdea1d + https://github.com/rust-lang/rust/pull/139493/files#diff-49849ff961ebc978f98448c8990cf7aae8e94cb03db44f016011aa8400170587.Coverage
A variety of UI tests including edge cases have been added.
Outstanding bugs
An old bug is made more noticeable by this change #145577 but it was recommended to not block on it #139493 (comment).
Outstanding FIXMEs
https://github.com/rust-lang/rust/pull/139493/files#diff-c046507afdba3b0705638f53fffa156cbad72ed17aa01d96d7bd1cc10b8d9bce
Tool changes
rustfmtrust-analyzerrustdoc (both JSON and HTML)cargoclippyrustupdocs.rsNo known changes needed or expected.
Breaking changes
Breaking changes:
It's possible for user code to invoke an ambiguity by defining their own macros with standard library names and glob importing them, e.g.
use nom::*importingnom::dbg. In practice this happens rarely based on crater data. The 3 public crates where this was an issue, have been fixed. The ambiguous panic import is more common and affects a non-trivial amount of the public - and likely private - crate ecosystem. To avoid a breaking change, a new future incompatible lint was added ambiguous_panic_imports see Tracking Issue for future-incompatibility lintambiguous_panic_imports#147319. This allows current code to continue compiling, albeit with a new warning. Future editions of Rust make this an error and future versions of Rust can choose to make this error. Technically this is a breaking change, but crater gives us the confidence that the impact will be at worst a new warning for 99+% of public and private crates.Code using
#![no_implicit_prelude]and Rust edition 2015 will no longer automatically have access to the prelude macros. The following works on nightly by would stop working with this change:Crater found no instance of this pattern in use. Affected code can fix the issue by directly importing the macros. The new behavior matches the behavior of
#![no_implicit_prelude]in Rust editions 2018 and beyond and it's intuitive meaning.Crater report:
Crater analysis:
PRs to affected crates:
Type system, opsem
Compile-time checks
N/A
Type system rules
N/A
Sound by default?
N/A
Breaks the AM?
N/A
Common interactions
Temporaries
N/A
Drop order
N/A
Pre-expansion / post-expansion
N/A
Edition hygiene
N/A
SemVer implications
No.
Exposing other features
No.
History
assert_matchesand move it tocore::macros#137487 (comment)Acknowledgments
More or less solo developed by @Voultapher with some help from @petrochenkov.
Open items
None.