Skip to content

Fix syntax error false positives for escapes and quotes in f-strings #20867

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Oct 15, 2025
Merged

Fix syntax error false positives for escapes and quotes in f-strings #20867

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
57a62b5
Fix syntax error false positives for escapes and quotes in f-strings
ntBre Oct 14, 2025
eabc697
add failing test cases
ntBre Oct 14, 2025
6e457d6
fix errors, accept snapshots
ntBre Oct 14, 2025
62df545
add some extra tests
ntBre Oct 14, 2025
b562fe0
accept formatter snapshots
ntBre Oct 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions crates/ruff_python_formatter/resources/test/fixtures/ruff/expression/fstring.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -706,8 +706,6 @@
f'{1: abcd "{1}" }'
f'{1: abcd "{'aa'}" }'
f'{1=: "abcd {'aa'}}'
# FIXME(brent) This should not be a syntax error on output. The escaped quotes are in the format
# spec, which is valid even before 3.12.
f'{x:a{z:hy "user"}} \'\'\''

# Changing the outer quotes is fine because the format-spec is in a nested expression.
Expand Down
54 changes: 0 additions & 54 deletions crates/ruff_python_formatter/tests/snapshots/format@expression__fstring.py.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -712,8 +712,6 @@ f'{1:hy "user"}'
f'{1: abcd "{1}" }'
f'{1: abcd "{'aa'}" }'
f'{1=: "abcd {'aa'}}'
# FIXME(brent) This should not be a syntax error on output. The escaped quotes are in the format
# spec, which is valid even before 3.12.
f'{x:a{z:hy "user"}} \'\'\''

# Changing the outer quotes is fine because the format-spec is in a nested expression.
Expand Down Expand Up @@ -1536,8 +1534,6 @@ f'{1:hy "user"}'
f'{1: abcd "{1}" }'
f'{1: abcd "{"aa"}" }'
f'{1=: "abcd {'aa'}}'
# FIXME(brent) This should not be a syntax error on output. The escaped quotes are in the format
# spec, which is valid even before 3.12.
f"{x:a{z:hy \"user\"}} '''"

# Changing the outer quotes is fine because the format-spec is in a nested expression.
Expand Down Expand Up @@ -2365,8 +2361,6 @@ f'{1:hy "user"}'
f'{1: abcd "{1}" }'
f'{1: abcd "{"aa"}" }'
f'{1=: "abcd {'aa'}}'
# FIXME(brent) This should not be a syntax error on output. The escaped quotes are in the format
# spec, which is valid even before 3.12.
f"{x:a{z:hy \"user\"}} '''"

# Changing the outer quotes is fine because the format-spec is in a nested expression.
Expand Down Expand Up @@ -2418,30 +2412,6 @@ print(f"{ {}, 1 }")


### Unsupported Syntax Errors
error[invalid-syntax]: Cannot use an escape sequence (backslash) in f-strings on Python 3.10 (syntax was added in Python 3.12)
--> fstring.py:764:19
|
762 | # FIXME(brent) This should not be a syntax error on output. The escaped quotes are in the format
763 | # spec, which is valid even before 3.12.
764 | f"{x:a{z:hy \"user\"}} '''"
| ^
765 |
766 | # Changing the outer quotes is fine because the format-spec is in a nested expression.
|
warning: Only accept new syntax errors if they are also present in the input. The formatter should not introduce syntax errors.

error[invalid-syntax]: Cannot use an escape sequence (backslash) in f-strings on Python 3.10 (syntax was added in Python 3.12)
--> fstring.py:764:13
|
762 | # FIXME(brent) This should not be a syntax error on output. The escaped quotes are in the format
763 | # spec, which is valid even before 3.12.
764 | f"{x:a{z:hy \"user\"}} '''"
| ^
765 |
766 | # Changing the outer quotes is fine because the format-spec is in a nested expression.
|
warning: Only accept new syntax errors if they are also present in the input. The formatter should not introduce syntax errors.

error[invalid-syntax]: Cannot reuse outer quote character in f-strings on Python 3.10 (syntax was added in Python 3.12)
--> fstring.py:178:8
|
Expand All @@ -2452,27 +2422,3 @@ error[invalid-syntax]: Cannot reuse outer quote character in f-strings on Python
179 | f"foo {'"bar"'}"
|
warning: Only accept new syntax errors if they are also present in the input. The formatter should not introduce syntax errors.

error[invalid-syntax]: Cannot reuse outer quote character in f-strings on Python 3.10 (syntax was added in Python 3.12)
--> fstring.py:773:14
|
771 | f'{1=: "abcd \'\'}' # Don't change the outer quotes, or it results in a syntax error
772 | f"{1=: abcd \'\'}" # Changing the quotes here is fine because the inner quotes aren't the opposite quotes
773 | f"{1=: abcd \"\"}" # Changing the quotes here is fine because the inner quotes are escaped
| ^
774 | # Don't change the quotes in the following cases:
775 | f'{x=:hy "user"} \'\'\''
|
warning: Only accept new syntax errors if they are also present in the input. The formatter should not introduce syntax errors.

error[invalid-syntax]: Cannot reuse outer quote character in f-strings on Python 3.10 (syntax was added in Python 3.12)
--> fstring.py:764:14
|
762 | # FIXME(brent) This should not be a syntax error on output. The escaped quotes are in the format
763 | # spec, which is valid even before 3.12.
764 | f"{x:a{z:hy \"user\"}} '''"
| ^
765 |
766 | # Changing the outer quotes is fine because the format-spec is in a nested expression.
|
warning: Only accept new syntax errors if they are also present in the input. The formatter should not introduce syntax errors.
2 changes: 2 additions & 0 deletions crates/ruff_python_parser/resources/inline/err/nested_quote_in_format_spec_py312.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# parse_options: {"target-version": "3.12"}
f"{1:""}" # this is a ParseError on all versions
2 changes: 2 additions & 0 deletions crates/ruff_python_parser/resources/inline/ok/non_nested_quote_in_format_spec_py311.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# parse_options: {"target-version": "3.11"}
f"{1:''}" # but this is okay on all versions
2 changes: 2 additions & 0 deletions crates/ruff_python_parser/resources/inline/ok/pep701_f_string_py311.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@
f"""{f'''# before expression {f'# aro{f"#{1+1}#"}und #'}'''} # after expression"""
f"escape outside of \t {expr}\n"
f"test\"abcd"
f"{1:\x64}" # escapes are valid in the format spec
f"{1:\"d\"}" # this also means that escaped outer quotes are valid
26 changes: 21 additions & 5 deletions crates/ruff_python_parser/src/parser/expression.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1569,6 +1569,8 @@ impl<'src> Parser<'src> {
// f"""{f'''# before expression {f'# aro{f"#{1+1}#"}und #'}'''} # after expression"""
// f"escape outside of \t {expr}\n"
// f"test\"abcd"
// f"{1:\x64}" # escapes are valid in the format spec
// f"{1:\"d\"}" # this also means that escaped outer quotes are valid

// test_err pep701_f_string_py311
// # parse_options: {"target-version": "3.11"}
Expand All @@ -1584,6 +1586,13 @@ impl<'src> Parser<'src> {
// f"""{f"""{x}"""}""" # mark the whole triple quote
// f"{'\n'.join(['\t', '\v', '\r'])}" # multiple escape sequences, multiple errors

// test_err nested_quote_in_format_spec_py312
// # parse_options: {"target-version": "3.12"}
// f"{1:""}" # this is a ParseError on all versions

// test_ok non_nested_quote_in_format_spec_py311
// # parse_options: {"target-version": "3.11"}
// f"{1:''}" # but this is okay on all versions
let range = self.node_range(start);

if !self.options.target_version.supports_pep_701()
Expand All @@ -1592,22 +1601,29 @@ impl<'src> Parser<'src> {
let quote_bytes = flags.quote_str().as_bytes();
let quote_len = flags.quote_len();
for expr in elements.interpolations() {
for slash_position in memchr::memchr_iter(b'\\', self.source[expr.range].as_bytes())
{
// We need to check the whole expression range, including any leading or trailing
// debug text, but exclude the format spec, where escapes and escaped, reused quotes
// are allowed.
let range = expr
.format_spec
.as_ref()
.map(|format_spec| TextRange::new(expr.start(), format_spec.start()))
.unwrap_or(expr.range);
for slash_position in memchr::memchr_iter(b'\\', self.source[range].as_bytes()) {
let slash_position = TextSize::try_from(slash_position).unwrap();
self.add_unsupported_syntax_error(
UnsupportedSyntaxErrorKind::Pep701FString(FStringKind::Backslash),
TextRange::at(expr.range.start() + slash_position, '\\'.text_len()),
TextRange::at(range.start() + slash_position, '\\'.text_len()),
);
}

if let Some(quote_position) =
memchr::memmem::find(self.source[expr.range].as_bytes(), quote_bytes)
memchr::memmem::find(self.source[range].as_bytes(), quote_bytes)
{
let quote_position = TextSize::try_from(quote_position).unwrap();
self.add_unsupported_syntax_error(
UnsupportedSyntaxErrorKind::Pep701FString(FStringKind::NestedQuote),
TextRange::at(expr.range.start() + quote_position, quote_len),
TextRange::at(range.start() + quote_position, quote_len),
);
}
}
Expand Down
90 changes: 90 additions & 0 deletions ...ff_python_parser/tests/snapshots/invalid_syntax@nested_quote_in_format_spec_py312.py.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
---
source: crates/ruff_python_parser/tests/fixtures.rs
input_file: crates/ruff_python_parser/resources/inline/err/nested_quote_in_format_spec_py312.py
---
## AST

```
Module(
ModModule {
node_index: NodeIndex(None),
range: 0..94,
body: [
Expr(
StmtExpr {
node_index: NodeIndex(None),
range: 44..53,
value: FString(
ExprFString {
node_index: NodeIndex(None),
range: 44..53,
value: FStringValue {
inner: Concatenated(
[
FString(
FString {
range: 44..50,
node_index: NodeIndex(None),
elements: [
Interpolation(
InterpolatedElement {
range: 46..49,
node_index: NodeIndex(None),
expression: NumberLiteral(
ExprNumberLiteral {
node_index: NodeIndex(None),
range: 47..48,
value: Int(
1,
),
},
),
debug_text: None,
conversion: None,
format_spec: Some(
InterpolatedStringFormatSpec {
range: 49..49,
node_index: NodeIndex(None),
elements: [],
},
),
},
),
],
flags: FStringFlags {
quote_style: Double,
prefix: Regular,
triple_quoted: false,
},
},
),
Literal(
StringLiteral {
range: 50..53,
node_index: NodeIndex(None),
value: "}",
flags: StringLiteralFlags {
quote_style: Double,
prefix: Empty,
triple_quoted: false,
},
},
),
],
),
},
},
),
},
),
],
},
)
```
## Errors

|
1 | # parse_options: {"target-version": "3.12"}
2 | f"{1:""}" # this is a ParseError on all versions
| ^ Syntax Error: f-string: expecting '}'
|
77 changes: 77 additions & 0 deletions ..._python_parser/tests/snapshots/valid_syntax@non_nested_quote_in_format_spec_py311.py.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
---
source: crates/ruff_python_parser/tests/fixtures.rs
input_file: crates/ruff_python_parser/resources/inline/ok/non_nested_quote_in_format_spec_py311.py
---
## AST

```
Module(
ModModule {
node_index: NodeIndex(None),
range: 0..90,
body: [
Expr(
StmtExpr {
node_index: NodeIndex(None),
range: 44..53,
value: FString(
ExprFString {
node_index: NodeIndex(None),
range: 44..53,
value: FStringValue {
inner: Single(
FString(
FString {
range: 44..53,
node_index: NodeIndex(None),
elements: [
Interpolation(
InterpolatedElement {
range: 46..52,
node_index: NodeIndex(None),
expression: NumberLiteral(
ExprNumberLiteral {
node_index: NodeIndex(None),
range: 47..48,
value: Int(
1,
),
},
),
debug_text: None,
conversion: None,
format_spec: Some(
InterpolatedStringFormatSpec {
range: 49..51,
node_index: NodeIndex(None),
elements: [
Literal(
InterpolatedStringLiteralElement {
range: 49..51,
node_index: NodeIndex(None),
value: "''",
},
),
],
},
),
},
),
],
flags: FStringFlags {
quote_style: Double,
prefix: Regular,
triple_quoted: false,
},
},
),
),
},
},
),
},
),
],
},
)
```
Loading