Initial work towards new semantic analyzer #6240
Conversation
The only different to `mypy.semanal_shared` for now is the change of the return type of `anal_type` to be optional.
| @@ -181,7 +181,7 @@ def anal_type(self, t: Type, *, | |||
| allow_tuple_literal: bool = False, | |||
| allow_unbound_tvars: bool = False, | |||
| report_invalid_types: bool = True, | |||
| third_pass: bool = False) -> Type: | |||
| third_pass: bool = False) -> Optional[Type]: | |||
There was a problem hiding this comment.
None means some names were nor resolved (type not ready), right? Maybe document this?
| @@ -0,0 +1,85 @@ | |||
| -- Test cases for the new semantic analyzer | |||
There was a problem hiding this comment.
I think we should start gradually adding existing tests to the new semantic analyzer whitelist soon, instead of adding new tests. In few first PRs it is OK however.
There was a problem hiding this comment.
Yeah, we can start thinking of this once basic stuff works.
| [file a.py] | ||
| from b import bad # E: Module 'b' has no attribute 'bad' | ||
| [file b.py] | ||
| from a import bad2 # E: Module 'a' has no attribute 'bad2' |
There was a problem hiding this comment.
Maybe also add a test where the both modules import the same name (bad in this case)?
| def semantic_analysis_for_scc(graph: 'Graph', scc: List[str]) -> None: | ||
| # Assume reachability analysis has already been performed. | ||
| process_top_levels(graph, scc) | ||
| process_functions(graph, scc) |
There was a problem hiding this comment.
I would be more comfortable with a TODO here (for either implementation, or documenting that it is not supported) about global declarations in functions like this:
def f():
global x
x = 1
print(x) # <- is this allowed?| deferred = [] # type: List[str] | ||
| while worklist: | ||
| next_id = worklist.pop() | ||
| deferred += semantic_analyze_target(next_id, graph[next_id]) |
There was a problem hiding this comment.
Just a suggestion: Maybe we can add a simple abstraction layer for adding to/removing from the deferred queue? Currently this can be just a function calling set operations, but will be easier to refactor if we later will decide to switch to priority queues or similar.
| targets = get_all_leaf_targets(symtable) | ||
| for target in targets: | ||
| deferred += semantic_analyze_target(target, graph[id]) | ||
| assert not deferred # There can't be cross-function forward refs |
There was a problem hiding this comment.
Is this a temporary limitation, right?
There was a problem hiding this comment.
It's not a limitation as such, since deferring functions should never be necessary. All namespaces need to complete before start processing functions.
| info._fullname = self.qualified_name(defn.name) | ||
| self.globals[defn.name] = SymbolTableNode(GDEF, info) | ||
| else: | ||
| info._fullname = info.name() |
There was a problem hiding this comment.
I assume TODO about nested classes also applies here.
| typ = t.accept(a) | ||
| if self.num_incomplete_refs != prev_incomplete: |
There was a problem hiding this comment.
This way looks a bit tricky, maybe we can get this somehow from the return type (or use an exception)? This is fine for a follow-up PR if you agree.
There was a problem hiding this comment.
Using return type may be tricky, since this needs to also work with the main AST visitor, which doesn't return anything. Using an exception may be a good idea. Let's revisit this later once we have a more complete implementation.
| result.append(analyzed) | ||
| else: | ||
| # TODO: Is this the right thing to do? | ||
| result.append(AnyType(TypeOfAny.from_error)) |
There was a problem hiding this comment.
Should we just return Optional[List[Type]] from this function instead?
There was a problem hiding this comment.
Maybe, it's unclear at this point. We can get back to this later.
|
I'm going to merge this as-is since I have some other work almost ready that is built on top of this. I'll address some of the comments in another PR, and for others we can create follow-up tasks. |
These respond to some review feedback to #6240.
These respond to some review feedback in #6240.
Implement basic deferred nodes functionality and remove most of old
pass 1 functionality. Also fork
mypy.semanal_sharedsince I neededto tweak the signature of
anal_typein an incompatible way (there arenot other changes in that module).
A lot of things still don't work, including functions and more than two
passes. There are a small number of tests for the new semantic
analyzer, but I made no attempt at achieving good test coverage yet.
Work towards #6204.