New semantic analyzer: infinite loop in daemon with decorators, nested classes

[msullivan]

Minimized from an example reported by a user at dropbox.

[case testInfiniteLoop]
# Only happens with new analyzer???
# flags: --new-semantic-analyzer
[file a.py]
from b import f
from typing import Callable, TypeVar

F = TypeVar('F', bound=Callable)
def dec(x: F) -> F: return x

@dec
def foo(self):
    class A:
        @classmethod
        def asdf(cls, x: 'A') -> None: pass

@dec
def bar(self):
    class B:
        @classmethod
        def asdf(cls, x: 'B') -> None: pass
    f()

[file b.py]
def f() -> int: pass
[file b.py.2]
def f() -> str: pass
[builtins fixtures/classmethod.pyi]
[out]
==

The issue seems to be that if a nested class isn't reprocessed and the toplevel gets reprocessed, it disappears from the symbol table and triggers its dependencies. If the toplevel depends on the nested class, then this can cause ping-ponging where the two nested classes alternate disappearing and reappearing.

The decorator and the method referencing the class are there because the create the needed dependencies.

[msullivan]

Another related case with NamedTuple

[case testInfiniteLoop2]
# Only happens with new analyzer???
# flags: --new-semantic-analyzer
[file a.py]
from b import f
from typing import Callable, TypeVar, NamedTuple

F = TypeVar('F', bound=Callable)
def dec(x: F) -> F: return x

@dec
def foo(self):
    N = NamedTuple('N', [('x', int)])
    def g(x: N) -> None: pass

@dec
def bar(self):
    N = NamedTuple('N', [('x', int)])
    def g(x: N) -> None: pass
    f()

[file b.py]
def f() -> int: pass
[file b.py.2]
def f() -> str: pass
[builtins fixtures/classmethod.pyi]
[out]
==

[ilevkivskyi]

This is essentially a duplicate of #6422 (see the last bullet about fine-grained mode). I can either add a hot fix, or we can try a more general fix.

[JukkaL]

How much work would a general fix be relative to a hot fix?

[ilevkivskyi]

How much work would a general fix be relative to a hot fix?

It's hard to tell, by I think it may be 2-3 days, the refactoring of this area is quite overdue.

[JukkaL]

Let's plan for that separately and implement a quick fix for now.

[ilevkivskyi]

OK.