Stabilize `use_extern_macros` #50911

petrochenkov · 2018-05-20T01:05:09Z

Closes #35896

rust-highfive · 2018-05-20T01:05:12Z

r? @cramertj

(rust_highfive has picked a reviewer for you, use r? to override)

petrochenkov · 2018-05-20T01:05:44Z

r? @kennytm

petrochenkov · 2018-05-20T01:05:57Z

@bors try

bors · 2018-05-20T01:06:13Z

⌛ Trying commit 66dfde18bfdfe1f9ae1c516e56228c77284a90b5 with merge ea32c49808849674e3d126b5732b2abfba9ed226...

bors · 2018-05-20T02:52:20Z

☀️ Test successful - status-travis
State: approved= try=True

kennytm · 2018-05-20T06:33:13Z

cc @rust-lang/infra Crater run (check only) requested.

pietroalbini · 2018-05-21T09:06:47Z

Crater run (check only) started.

pietroalbini · 2018-05-24T07:14:56Z

Hi @kennytm (crater requester, PR reviewer)! Crater results are at: http://cargobomb-reports.s3.amazonaws.com/pr-50911/index.html. 'Blacklisted' crates (spurious failures etc) can be found here. If you see any spurious failures not on the list, please make a PR against that file.

(interested observers: Crater is a tool for testing the impact of changes on the crates.io ecosystem. You can find out more at the repo if you're curious)

kennytm · 2018-05-24T08:22:33Z

6 regressions.

verilog: Seems to be spurious ("Read-only file system" cc @pietroalbini).
npbot: This PR seems to have broken its use of structopt-^0.2.5 via quicli-0.2.0. Legit.
The rest: All due to #[macro_use] no longer needed, which causes #[deny(warnings)] to error out. Legit but expected.

Relevant source code in npbot-1.0.0

[dependencies]
structopt = "0.2.5"

#[macro_use]
extern crate quicli;

// ...

use structopt::StructOpt;
use quicli::prelude::*;

// ...

#[derive(StructOpt)]
pub(crate) struct Opt {
    #[structopt(short = "d", long = "debug", help = "only use the local, println!() sink")]
    debug: bool,
    #[structopt(long = "verbose", short = "v", parse(from_occurrences))]
    verbose: u8,

    // for the SoundTouch source
    #[structopt(long = "soundtouch-host")]
    soundtouch_host: Option<String>,

    // for the last.fm source
    #[structopt(long = "lastfm-api-key")]
    lastfm_api_key: Option<String>,
    #[structopt(long = "lastfm-username")]
    lastfm_username: Option<String>,
}

Note the lack of #[macro_use] extern crate structopt.

Error log

May 22 22:49:57.928 INFO kablam! error[E0658]: The attribute `structopt` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
May 22 22:49:57.928 INFO kablam!   --> src/main.rs:41:5
May 22 22:49:57.928 INFO kablam!    |
May 22 22:49:57.928 INFO kablam! 41 |     #[structopt(short = "d", long = "debug", help = "only use the local, println!() sink")]
May 22 22:49:57.928 INFO kablam!    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 22 22:49:57.928 INFO kablam!    |
May 22 22:49:57.929 INFO kablam!    = help: add #![feature(custom_attribute)] to the crate attributes to enable
May 22 22:49:57.929 INFO kablam! 
May 22 22:49:57.929 INFO kablam! error[E0658]: The attribute `structopt` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
May 22 22:49:57.929 INFO kablam!   --> src/main.rs:43:5
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam! 43 |     #[structopt(long = "verbose", short = "v", parse(from_occurrences))]
May 22 22:49:57.929 INFO kablam!    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam!    = help: add #![feature(custom_attribute)] to the crate attributes to enable
May 22 22:49:57.929 INFO kablam! 
May 22 22:49:57.929 INFO kablam! error[E0658]: The attribute `structopt` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
May 22 22:49:57.929 INFO kablam!   --> src/main.rs:47:5
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam! 47 |     #[structopt(long = "soundtouch-host")]
May 22 22:49:57.929 INFO kablam!    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam!    = help: add #![feature(custom_attribute)] to the crate attributes to enable
May 22 22:49:57.929 INFO kablam! 
May 22 22:49:57.929 INFO kablam! error[E0658]: The attribute `structopt` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
May 22 22:49:57.929 INFO kablam!   --> src/main.rs:51:5
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam! 51 |     #[structopt(long = "lastfm-api-key")]
May 22 22:49:57.929 INFO kablam!    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam!    = help: add #![feature(custom_attribute)] to the crate attributes to enable
May 22 22:49:57.929 INFO kablam! 
May 22 22:49:57.930 INFO kablam! error[E0658]: The attribute `structopt` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
May 22 22:49:57.930 INFO kablam!   --> src/main.rs:53:5
May 22 22:49:57.930 INFO kablam!    |
May 22 22:49:57.930 INFO kablam! 53 |     #[structopt(long = "lastfm-username")]
May 22 22:49:57.930 INFO kablam!    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 22 22:49:57.930 INFO kablam!    |
May 22 22:49:57.930 INFO kablam!    = help: add #![feature(custom_attribute)] to the crate attributes to enable
May 22 22:49:57.930 INFO kablam! 
May 22 22:49:57.931 INFO kablam! error: aborting due to 5 previous errors

petrochenkov · 2018-05-24T10:09:31Z

I'll look what happens with npbot.

petrochenkov · 2018-05-27T19:13:35Z

src/librustc_resolve/macros.rs

            MacroBinding::Global(binding)
        } else {
            return None;
        };

-        if !self.use_extern_macros {
-            if let Some(scope) = possible_time_travel {


FIXME: possible_time_travel is now unused

petrochenkov · 2018-05-28T15:55:42Z

It looks like npbot legitimately encounters a case of indeterminate resolution when macro expansion cannot progress and generates dummy fragments to recover.

So the error should be something like "cannot determine resolution for the derive macro StructOpt".

However, due to a scary hole in our macro resolution checking such dummy expansions can happen silently without generating any errors.
So we are seeing only secondary errors about unknown structopt attributes instead of the actual error about unresolved StructOpt.
I'll prepare a fix.

petrochenkov · 2018-05-29T01:53:09Z

Fixed in #51145

petrochenkov · 2018-05-30T14:54:24Z

r? @alexcrichton

resolve: Some macro resolution refactoring Work towards completing #50911 (comment) The last commit also fixes #53269 by not using `def_id()` on `Def::Err` and also fixes #53512.

resolve: Future proof resolutions for potentially built-in attributes Based on #53778 This is not full "pass all attributes through name resolution", but a more conservative solution. If built-in attribute is ambiguous with any other macro in scope, then an error is reported. TODO: Explain what complications arise with the full solution. cc #50911 (comment) Closes #53531

Feature gate non-builtin attributes in inner attribute position Closes item 3 from #50911 (comment)

resolve: Future proof resolutions for potentially built-in attributes This is not full "pass all attributes through name resolution", but a more conservative solution. If built-in attribute is ambiguous with any other macro in scope, then an error is reported. What complications arise with the full solution - #53913 (comment). cc #50911 (comment) cc #52269 Closes #53531

@ghost

Temporarily prohibit proc macro attributes placed after derives ... and also proc macro attributes used together with `#[test]`/`#[bench]`. Addresses item 6 from #50911 (comment). The end goal is straightforward predictable left-to-right expansion order for attributes. Right now derives are expanded last regardless of their relative ordering with macro attributes and right now it's simpler to temporarily prohibit macro attributes placed after derives than changing the expansion order. I'm not sure whether the new beta is already released or not, but if it's released, then this patch needs to be backported, so the solution needs to be minimal. How to fix broken code (derives): - Move macro attributes above derives. This won't change expansion order, they are expanded before derives anyway. Using attribute macros on same items with `#[test]` and `#[bench]` is prohibited for similar expansion order reasons, but this one is going to be reverted much sooner than restrictions on derives. How to fix broken code (test/bench): - Enable `#![feature(plugin)]` (don't ask why). r? @ghost

softprops · 2018-10-26T14:39:55Z

The 1.30.0 release notes for #[macro_export(local_inner_macros)] link here.

Where can I find more documentation on this feature?

So far I've tried local_inner_macros in two cases where a crate's macros depended on anothers and it didn't work in either case.

iliana/rust-crowbar#46 (comment)

dtolnay/mashup#16

petrochenkov · 2018-10-26T14:44:32Z

@softprops
rust-lang/edition-guide#117

It will be available at https://rust-lang-nursery.github.io/edition-guide/rust-2018/macros/macro-changes.html when merged.

softprops · 2018-10-26T15:11:56Z

@petrochenkov thanks! I'll take a look and report back if it looks like what I'm trying to do, re-export transitive dependency crates macros` is actually what this new feature supports.

softprops · 2018-10-26T15:41:17Z

@petrochenkov l need to do a bit more testing but I worked it out, at least enough to get a fork of rust crowbar to compile. I still need to test a crate that uses that fork before I can call this 👍 'd

It may not be clear to new rust users that for foreign macros, you'll also need namespace the macro's path. In my case I just needed to prefix crowbars use of cpython's macros with cpython:: in the docs you linked to and users will find you only use local macros in examples using $crate::. Assuming macros authors are already going to be a bit more advanced than the beginner rust users they may be able to discover this for themselves. I'd recommend adding at least one foreign macro example for the sake of beginner/advanced user inclusivity. Albeit this would be much more painful for the example of supporting old rust versions because if forces users to copy and paste the entire macro srcs from other crates. The format_args is probably the simplest case for a macro. A more complex examples will be much more awkward. For now, asking users to use the latest stable works well enough!

softprops · 2018-10-26T16:25:05Z

found a potential bug but I'm not sure if its by design

when experimenting with the $crate:: path prefix with #[macro_export(local_inner_macros)] and with #[macro_export] I found this interesting inconsistency gotcha with the way that exported macro can be used.

with #[macro_export(local_inner_macros)] I can do the following

extern crate crowbar;
crowbar::lambda!(...)

however with #[macro_export] I get the following here using the same path references

extern crate crowbar;
crowbar::lambda!(...)

error: cannot find macro `lambda!` in this scope

interestingly though I can do the following with #[macro_export]

extern crate crowbar;
use crowbar::lambda;
lambda!(...)

this feels a bit inconsistent in the way the newer system attempts to line up with what you can do with path references for non macro items.

petrochenkov · 2018-10-26T16:43:02Z

@softprops
I suspect that the lambda macro is recursive and calls itself by relative name (lambda!(...)), in other words, it's its own helper.

This way the first iteration crowbar::lambda!(...) is resolved successfully, but the next iterations cannot be resolved without bringing lambda in scope at use-site or using local_inner_macros at def-site.

softprops · 2018-10-26T17:13:59Z

@petrochenkov 100%. that was it. thanks!

Signed-off-by: JmPotato <ghzpotato@gmail.com> ### What problem does this PR solve? Import macros from other crates with the `use` syntax. ### What is changed and how it works? rust-lang/rust#50911 ### Check List  Tests  - Unit test - Integration test ### Release note  * No release note.

rust-highfive assigned cramertj May 20, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 20, 2018

petrochenkov added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 20, 2018

rust-highfive assigned kennytm and unassigned cramertj May 20, 2018

aturon mentioned this pull request May 22, 2018

Tracking issue for RFC 1566: Procedural macros #38356

Closed

31 tasks

pietroalbini added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels May 24, 2018

kennytm added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 24, 2018

petrochenkov commented May 27, 2018

View reviewed changes

petrochenkov mentioned this pull request May 29, 2018

resolve: Make sure indeterminate and inconsistent macro resolutions always generate errors #51145

Merged

petrochenkov force-pushed the macuse branch from 66dfde1 to df02c89 Compare May 30, 2018 14:53

petrochenkov changed the title ~~[Do not merge] Stabilize use_extern_macros~~ Stabilize use_extern_macros May 30, 2018

rust-highfive unassigned kennytm May 30, 2018

This was referenced Aug 19, 2018

resolve: Some macro resolution refactoring #53471

Merged

Disallow attribute macros named cfg #53531

Closed

LukasKalbertodt mentioned this pull request Aug 22, 2018

Guard unstable proc_macro things behind nightly feature auto-impl-rs/auto_impl#26

Merged

This was referenced Sep 2, 2018

Expand Procedural Macro Docs rust-lang/reference#412

Merged

resolve: Future proof resolutions for potentially built-in attributes #53913

Merged

AnneKitsune mentioned this pull request Sep 9, 2018

Specs-derive amethyst/specs#452

Closed

petrochenkov mentioned this pull request Sep 10, 2018

Feature gate non-builtin attributes in inner attribute position #54093

Merged

bors added a commit that referenced this pull request Sep 10, 2018

Auto merge of #54093 - petrochenkov:noinner, r=alexcrichton

551244f

Feature gate non-builtin attributes in inner attribute position Closes item 3 from #50911 (comment)

petrochenkov mentioned this pull request Sep 16, 2018

Temporarily prohibit proc macro attributes placed after derives #54277

Merged

ghost mentioned this pull request Oct 26, 2018

Rust 1.30: importing derive macro with use works or fails depending on whether unrelated dependency is present #55386

Closed

carols10cents mentioned this pull request Nov 2, 2018

clarify discussion of macros and namspaces rust-lang/book#1585

Closed

rye mentioned this pull request Dec 23, 2018

Stop using outdated feature gates CodeSandwich/Mocktopus#31

Merged

petrochenkov deleted the macuse branch June 5, 2019 16:16

JmPotato mentioned this pull request Feb 13, 2021

pd_client: refine PD client codes tikv/tikv#9651

Merged

Max-Cheng mentioned this pull request Feb 14, 2021

diagnostics:cpu hardware info add cpu arch tikv/tikv#9652

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stabilize `use_extern_macros` #50911

Stabilize `use_extern_macros` #50911

petrochenkov commented May 20, 2018 •

edited

Loading

rust-highfive commented May 20, 2018

petrochenkov commented May 20, 2018

petrochenkov commented May 20, 2018

bors commented May 20, 2018

bors commented May 20, 2018

kennytm commented May 20, 2018 •

edited

Loading

pietroalbini commented May 21, 2018

pietroalbini commented May 24, 2018

kennytm commented May 24, 2018 •

edited

Loading

petrochenkov commented May 24, 2018

petrochenkov May 27, 2018

petrochenkov commented May 28, 2018

petrochenkov commented May 29, 2018

petrochenkov commented May 30, 2018

softprops commented Oct 26, 2018

petrochenkov commented Oct 26, 2018

softprops commented Oct 26, 2018

softprops commented Oct 26, 2018 •

edited

Loading

softprops commented Oct 26, 2018

petrochenkov commented Oct 26, 2018 •

edited

Loading

softprops commented Oct 26, 2018

Stabilize use_extern_macros #50911

Stabilize use_extern_macros #50911

Conversation

petrochenkov commented May 20, 2018 • edited Loading

rust-highfive commented May 20, 2018

petrochenkov commented May 20, 2018

petrochenkov commented May 20, 2018

bors commented May 20, 2018

bors commented May 20, 2018

kennytm commented May 20, 2018 • edited Loading

pietroalbini commented May 21, 2018

pietroalbini commented May 24, 2018

kennytm commented May 24, 2018 • edited Loading

petrochenkov commented May 24, 2018

petrochenkov May 27, 2018

Choose a reason for hiding this comment

petrochenkov commented May 28, 2018

petrochenkov commented May 29, 2018

petrochenkov commented May 30, 2018

softprops commented Oct 26, 2018

petrochenkov commented Oct 26, 2018

softprops commented Oct 26, 2018

softprops commented Oct 26, 2018 • edited Loading

softprops commented Oct 26, 2018

petrochenkov commented Oct 26, 2018 • edited Loading

softprops commented Oct 26, 2018

Stabilize `use_extern_macros` #50911

Stabilize `use_extern_macros` #50911

petrochenkov commented May 20, 2018 •

edited

Loading

kennytm commented May 20, 2018 •

edited

Loading

kennytm commented May 24, 2018 •

edited

Loading

softprops commented Oct 26, 2018 •

edited

Loading

petrochenkov commented Oct 26, 2018 •

edited

Loading