Consider moving out-of-line module loading from parser to expansion #64197
Labels
A-frontend
Area: Compiler frontend (errors, parsing and HIR)
A-macros
Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..)
A-parser
Area: The parsing of Rust source code to an AST
T-compiler
Relevant to the compiler team, which will review and decide on the PR/issue.
Comments
petrochenkov
added
A-frontend
|
This is on my todo list, as it is a per-requisite for moving parser into a library. |
Merged
|
The PR trying to split the libsyntax crate apart (#65324) has to use some hacks due to cfg-expansion being called from the parser. |
|
#66565 removes those hacks in favor of a less principled approach (revealing the tight coupling between cfg expansion and parsing). |
|
I would like the ability to start parsing immediately when we see a |
Dylan-DPC-zz
pushed a commit
to Dylan-DPC-zz/rust
that referenced
this issue
Feb 13, 2020
expand: misc cleanups and simplifications Some work I did while trying to understand expand for the purposes of rust-lang#64197. r? @petrochenkov
Centril
added a commit
to Centril/rust
that referenced
this issue
Mar 10, 2020
…ov,eddyb
Expansion-driven outline module parsing
After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid:
```rust
#[cfg(FALSE)]
mod foo {
mod bar {
mod baz; // `foo/bar/baz.rs` doesn't exist, but no error!
}
}
```
Fixes rust-lang#64197.
r? @petrochenkov
Centril
added a commit
to Centril/rust
that referenced
this issue
Mar 10, 2020
…ov,eddyb
Expansion-driven outline module parsing
After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid:
```rust
#[cfg(FALSE)]
mod foo {
mod bar {
mod baz; // `foo/bar/baz.rs` doesn't exist, but no error!
}
}
```
Fixes rust-lang#64197.
r? @petrochenkov
Centril
added a commit
to Centril/rust
that referenced
this issue
Mar 11, 2020
…ov,eddyb
Expansion-driven outline module parsing
After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid:
```rust
#[cfg(FALSE)]
mod foo {
mod bar {
mod baz; // `foo/bar/baz.rs` doesn't exist, but no error!
}
}
```
Fixes rust-lang#64197.
r? @petrochenkov
Centril
added a commit
to Centril/rust
that referenced
this issue
Mar 16, 2020
Expansion-driven outline module parsing
After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid:
```rust
#[cfg(FALSE)]
mod foo {
mod bar {
mod baz; // `foo/bar/baz.rs` doesn't exist, but no error!
}
}
```
Fixes rust-lang#64197.
r? @petrochenkov
Centril
added a commit
to Centril/rust
that referenced
this issue
Mar 17, 2020
Expansion-driven outline module parsing
After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid:
```rust
#[cfg(FALSE)]
mod foo {
mod bar {
mod baz; // `foo/bar/baz.rs` doesn't exist, but no error!
}
}
```
Fixes rust-lang#64197.
r? @petrochenkov
Centril
added a commit
to Centril/rust
that referenced
this issue
Mar 17, 2020
Expansion-driven outline module parsing
After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid:
```rust
#[cfg(FALSE)]
mod foo {
mod bar {
mod baz; // `foo/bar/baz.rs` doesn't exist, but no error!
}
}
```
Fixes rust-lang#64197.
r? @petrochenkov
Dylan-DPC-zz
pushed a commit
to Dylan-DPC-zz/rust
that referenced
this issue
May 16, 2020
…e, r=petrochenkov Cleanup stale 'FIXME(rust-lang#64197)' (My first PR in rust-lang, any feedback is welcome. Please don't hesitate to let me know if I'm trying to do something pointless!) Trivial cleanup of a stale `FIXME`, `StringReader.pos` is no longer exposed. For testing added `pos()` method that returns cloned value of `pos`.
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
May 16, 2020
Rollup of 9 pull requests Successful merges: - rust-lang#71662 (Implement FromStr for OsString) - rust-lang#71677 (Add explicit references to the BuildHasher trait) - rust-lang#71724 (Doc alias improvements) - rust-lang#71948 (Suggest to await future before ? operator) - rust-lang#72090 (rustc_driver: factor out computing the exit code) - rust-lang#72206 (Cleanup stale 'FIXME(rust-lang#64197)') - rust-lang#72218 (make sure even unleashed miri does not do pointer stuff) - rust-lang#72220 ([const-prop] Don't replace Rvalues that are already constants) - rust-lang#72224 (doc: add links to rotate_(left|right)) Failed merges: r? @ghost
nnethercote
added a commit
to nnethercote/rust
that referenced
this issue
May 3, 2024
There are some test cases involving `parse` and `tokenstream` and `mut_visit` that are located in `rustc_expand`. Because it used to be the case that constructing a `ParseSess` required the involvement of `rustc_expand`. However, since rust-lang#64197 merged (a long time ago) `rust_expand` no longer needs to be involved. This commit moves the tests into `rustc_parse`. This is the optimal place for the `parse` tests. It's not ideal for the `tokenstream` and `mut_visit` tests -- they would be better in `rustc_ast` -- but they still rely on parsing, which is not available in `rustc_ast`. But `rustc_parse` is lower down in the crate graph and closer to `rustc_ast` than `rust_expand`, so it's still an improvement for them. The exact renaming is as follows: - rustc_expand/src/mut_visit/tests.rs -> rustc_parse/src/parser/mut_visit/tests.rs - rustc_expand/src/tokenstream/tests.rs -> rustc_parse/src/parser/tokenstream/tests.rs - rustc_expand/src/tests.rs + rustc_expand/src/parse/tests.rs -> compiler/rustc_parse/src/parser/tests.rs The latter two test files are combined because there's no need for them to be separate, and having a `rustc_parse::parser::parse` module would be weird. This also means some `pub(crate)`s can be removed.
nnethercote
added a commit
to nnethercote/rust
that referenced
this issue
May 6, 2024
There are some test cases involving `parse` and `tokenstream` and `mut_visit` that are located in `rustc_expand`. Because it used to be the case that constructing a `ParseSess` required the involvement of `rustc_expand`. However, since rust-lang#64197 merged (a long time ago) `rust_expand` no longer needs to be involved. This commit moves the tests into `rustc_parse`. This is the optimal place for the `parse` tests. It's not ideal for the `tokenstream` and `mut_visit` tests -- they would be better in `rustc_ast` -- but they still rely on parsing, which is not available in `rustc_ast`. But `rustc_parse` is lower down in the crate graph and closer to `rustc_ast` than `rust_expand`, so it's still an improvement for them. The exact renaming is as follows: - rustc_expand/src/mut_visit/tests.rs -> rustc_parse/src/parser/mut_visit/tests.rs - rustc_expand/src/tokenstream/tests.rs -> rustc_parse/src/parser/tokenstream/tests.rs - rustc_expand/src/tests.rs + rustc_expand/src/parse/tests.rs -> compiler/rustc_parse/src/parser/tests.rs The latter two test files are combined because there's no need for them to be separate, and having a `rustc_parse::parser::parse` module would be weird. This also means some `pub(crate)`s can be removed.
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
May 6, 2024
…ler-errors Move some tests from `rustc_expand` to `rustc_parse`. There are some test cases involving `parse` and `tokenstream` and `mut_visit` that are located in `rustc_expand`. Because it used to be the case that constructing a `ParseSess` required the involvement of `rustc_expand`. However, since rust-lang#64197 merged (a long time ago) `rust_expand` no longer needs to be involved. This commit moves the tests into `rustc_parse`. This is the optimal place for the `parse` tests. It's not ideal for the `tokenstream` and `mut_visit` tests -- they would be better in `rustc_ast` -- but they still rely on parsing, which is not available in `rustc_ast`. But `rustc_parse` is lower down in the crate graph and closer to `rustc_ast` than `rust_expand`, so it's still an improvement for them. The exact renaming is as follows: - rustc_expand/src/mut_visit/tests.rs -> rustc_parse/src/parser/mut_visit/tests.rs - rustc_expand/src/tokenstream/tests.rs -> rustc_parse/src/parser/tokenstream/tests.rs - rustc_expand/src/tests.rs + rustc_expand/src/parse/tests.rs -> compiler/rustc_parse/src/parser/tests.rs The latter two test files are combined because there's no need for them to be separate, and having a `rustc_parse::parser::parse` module would be weird. This also means some `pub(crate)`s can be removed. r? `@compiler-errors`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Due to module loading parser has to do some not very appropriate for a parser things like
cfgs on modules (duplicated with expansion)We also don't really need to load those files until we need to incorporate their contents into
NodeId-entified AST, which happens during expansion.The move will also mean never loading files in unconfigured code.
The loading is currently skipped only if
cfgis placed directly on themod m;, but not in other cases likeThe text was updated successfully, but these errors were encountered: