You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
During type inference on some invalid syntax examples, we encounter situations where the exact same expression appears in multiple places inside the AST. The smallest example I could find is this:
for
When parsing it, we get the following AST. Note that the same Name(…) expression (same range) appears both as target and as iter field on StmtFor, once with Store context, once with Invalid context:
This leads to a subsequent failing double-store assertion in store_expression_type because we infer types for both the target and the iter expressions:
let expr_id = expression.scoped_ast_id(self.db,self.scope());
let previous = self.types.expressions.insert(expr_id, ty);
assert_eq!(previous,None);
}
I'm not sure how to fix this.
Skipping storage when encountering expressions with Invalid context would work for this example, but there are other cases (:x=) where such an invalid expression is not "backed up" by a second valid expression.
Ignoring double-storage for Invalid expressions is also not an option, since in this example here, we run inference on iter first.
The text was updated successfully, but these errors were encountered:
Mentioning it here because it leads to a slightly different panic (Calling self.infer_expression on a standalone-expression is not allowed…). But the underlying issue is the same, I believe.
## Summary
Use the memory address to uniquely identify AST nodes, instead of
relying on source range and kind. The latter fails for ASTs resulting
from invalid syntax examples. See #14313 for details.
Also results in a 1-2% speedup
(https://codspeed.io/astral-sh/ruff/runs/67349cf55f36b36baa211360)
closes#14313
## Review
Here are the places where we use `NodeKey` directly or indirectly (via
`ExpressionNodeKey` or `DefinitionNodeKey`):
```rs
// semantic_index.rs
pub(crate) struct SemanticIndex<'db> {
// [...]
/// Map expressions to their corresponding scope.
scopes_by_expression: FxHashMap<ExpressionNodeKey, FileScopeId>,
/// Map from a node creating a definition to its definition.
definitions_by_node: FxHashMap<DefinitionNodeKey, Definition<'db>>,
/// Map from a standalone expression to its [`Expression`] ingredient.
expressions_by_node: FxHashMap<ExpressionNodeKey, Expression<'db>>,
// [...]
}
// semantic_index/builder.rs
pub(super) struct SemanticIndexBuilder<'db> {
// [...]
scopes_by_expression: FxHashMap<ExpressionNodeKey, FileScopeId>,
definitions_by_node: FxHashMap<ExpressionNodeKey, Definition<'db>>,
expressions_by_node: FxHashMap<ExpressionNodeKey, Expression<'db>>,
}
// semantic_index/ast_ids.rs
pub(crate) struct AstIds {
/// Maps expressions to their expression id.
expressions_map: FxHashMap<ExpressionNodeKey, ScopedExpressionId>,
/// Maps expressions which "use" a symbol (that is, [`ast::ExprName`]) to a use id.
uses_map: FxHashMap<ExpressionNodeKey, ScopedUseId>,
}
pub(super) struct AstIdsBuilder {
expressions_map: FxHashMap<ExpressionNodeKey, ScopedExpressionId>,
uses_map: FxHashMap<ExpressionNodeKey, ScopedUseId>,
}
```
## Test Plan
Added two failing examples to the corpus.
During type inference on some invalid syntax examples, we encounter situations where the exact same expression appears in multiple places inside the AST. The smallest example I could find is this:
for
When parsing it, we get the following AST. Note that the same
Name(…)
expression (same range) appears both astarget
and asiter
field onStmtFor
, once withStore
context, once withInvalid
context:This leads to a subsequent failing double-store assertion in
store_expression_type
because we infer types for both thetarget
and theiter
expressions:ruff/crates/red_knot_python_semantic/src/types/infer.rs
Lines 2176 to 2184 in f789b12
I'm not sure how to fix this.
Skipping storage when encountering expressions with
Invalid
context would work for this example, but there are other cases (:x=
) where such an invalid expression is not "backed up" by a second valid expression.Ignoring double-storage for
Invalid
expressions is also not an option, since in this example here, we run inference oniter
first.The text was updated successfully, but these errors were encountered: