Skip to content

Commit

Permalink
Make ' a forbidden follower for the RawLifetimeOrLabel rule
Browse files Browse the repository at this point in the history
This mirrors the changes from
rust-lang/rust#132341
which have been backported to appear in Rust 1.83
  • Loading branch information
mattheww committed Nov 25, 2024
1 parent c271568 commit 93b3775
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 3 deletions.
7 changes: 4 additions & 3 deletions src/lexlucid/pretokenisation/pretokenisation_rules.rs
Original file line number Diff line number Diff line change
Expand Up @@ -224,9 +224,9 @@ fn make_named_rules() -> BTreeMap<RuleName, Rule> {
)
"##)),

// Lifetime or label
// Raw lifetime or label
(RuleName::RawLifetimeOrLabel2021,
Rule::new_regex(
Rule::new_regex_with_forbidden_follower(
|cp| PretokenData::RawLifetimeOrLabel {
name: cp["name"].into(),
}, r##"\A
Expand All @@ -235,7 +235,8 @@ fn make_named_rules() -> BTreeMap<RuleName, Rule> {
[ \p{XID_Start} _ ]
\p{XID_Continue} *
)
"##)),
"##,
|c| c == '\'')),

// Reserved lifetime or label prefix
(RuleName::ReservedLifetimeOrLabelPrefix2021,
Expand Down
1 change: 1 addition & 0 deletions writeup/rationale.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ rather than trying to write non-overlapping lexer rules as the Reference does.
The model described in this document uses one-character lookahead (beyond the token which will be matched) in the prelexing step, in two cases:

- the lifetime-or-label rule, to prevent `'ab'c'` being analysed as `'ab` followed by `'c`
(and similarly for the raw-lifetime-or-label rule)
- the rule for float literals ending in `.`, to make sure that `1.a` is analysed as `1` `.` `a` rather than `1.` `a`

I think some kind of lookahead is unavoidable in these cases.
Expand Down
4 changes: 4 additions & 0 deletions writeup/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,10 @@ when `character_sequence` represents an iterator over the sequence of characters
)
```

Forbidden followers:

- The character <b>'</b>

##### Pretoken kind
`RawLifetimeOrLabel`

Expand Down

0 comments on commit 93b3775

Please sign in to comment.