Custom derives: TokenStream::from_str fails if the string contains 'mod foo;'

[comex]

It returns the somewhat unhelpful LexError { _inner: () }, which is caused by a ModulePathError in the parser. This happens regardless of whether an appropriately-named Rust file exists. If a path attribute is present and the path is not absolute, from_str still doesn't work, but panics after printing a proper error message rather than returning an error.

If macros 1.1 is used as intended, this is unlikely to come up. While the input (the body of a struct or enum) can contain a mod declaration inside a block inside an enum raw value, array length, etc., such declarations will generally already be expanded with the contents of the module file by the time they reach the custom derive - unless they are inside a macro invocation, in which case (if the derive preserves the invocation in its output) from_str will only parse it to a Mac, and the compiler will successfully deal with it later on, when it expands that macro. The questionable behavior only comes up if the custom derive either strips out a macro invocation or supplies its own code altogether.

(I ran across this when using macros 1.1 for a rather hackier purpose.)

Simple test case:

#![feature(rustc_macro, rustc_macro_lib)] 
extern crate rustc_macro;
use rustc_macro::TokenStream;
use std::str::FromStr;

#[rustc_macro_derive(foo)]
pub fn foo(_: TokenStream) -> TokenStream {
    TokenStream::from_str("mod foo;").unwrap()
}

[alexcrichton]

Added to tracking issue #35900

[jyn514]

This works fine on nightly (once you add some cargo crates and change rustc_macro to proc_macro):

error[E0583]: file not found for module `foo`
 --> src/main.rs:3:10
  |
3 | #[derive(foo)]
  |          ^^^
  |
  = help: to create the module `foo`, create file "src/foo.rs" or "src/foo/mod.rs"
  = note: this error originates in the derive macro `foo` (in Nightly builds, run with -Z macro-backtrace for more info)