Error emitted because object is inferred as having `Element[Literal["testsuites"]]` when `Element[str]` was expected

[Guitaricet]

Summary

Thank you so much for ty, it's already much better than the alternatives in both speed and ease of setup.

The error I'm seeing suggests that ty currently does not resolve Literal values into their actual types.

E.g., ty fails this code

import xml.etree.ElementTree as ET

testsuites = ET.Element("testsuites")
testsuite = ET.SubElement(testsuites, "testsuite")
# ^^^^^^^^^^ Expected `Element[str]`, found `Element[Literal["testsuites"]]`

Full output:

❯❯❯ ty check ty_error.py
error: lint:invalid-argument-type: Argument to this function is incorrect
 --> ty_error.py:4:27
  |
3 | testsuites = ET.Element("testsuites")
4 | testsuite = ET.SubElement(testsuites, "testsuite")
  |                           ^^^^^^^^^^ Expected `Element[str]`, found `Element[Literal["testsuites"]]`
5 | # ^^^^^^^^^^ Expected `Element[str]`, found `Element[Literal["testsuites"]]`
  |
info: Function defined here
   --> stdlib/xml/etree/ElementTree.pyi:140:5
    |
138 |     def __bool__(self) -> bool: ...
139 |
140 | def SubElement(parent: Element, tag: str, attrib: dict[str, str] = ..., **extra: str) -> Element: ...
    |     ^^^^^^^^^^ --------------- Parameter declared here
141 | def Comment(text: str | None = None) -> _CallableElement: ...
142 | def ProcessingInstruction(target: str, text: str | None = None) -> _CallableElement: ...
    |
info: `lint:invalid-argument-type` is enabled by default

Found 1 diagnostic

Expected behavior: ty should not fail this code.
Tested it with both ty-0.0.0a7 and ty-0.0.0a8 (commit 994caa6fcc2fdc7ab5e56dd1b86e3075aa404b9a)

Version

No response

[AlexWaygood]

Although Literal["testsuites"] is assignable to str, ty is correct in thinking that Element[Literal["testsuites"]] is not assignable to Element[str] That's because Element is generic over an invariant TypeVar in typeshed's stubs (and typeshed's stubs are what ty uses as the sole source of truth for types in the standard library):

https://github.com/python/typeshed/blob/41b2f9e7b4f0494b26052552d590886ed524b4aa/stdlib/xml/etree/ElementTree.pyi#L86-L89

So the issue here is the fact that we infer the testsuites variable as being of type Element[Literal["testsuites"]] at all -- we need to avoid that, and instead infer it as some type that is assignable to Element[str]. There are several ways in which we could accomplish this:

• Since no explicit type annotation was provided for testsuites, we could infer the type of the variable as being Element[Unknown | Literal["testsuites"]], which would be assignable to Element[str].
• We could apply a general principle that we should always upcast Literal types when inferring specialisations (I believe that is what both mypy and pyright are doing here); this would mean that we would infer Element[str] for the variable
• We could do something similar to what Rust does in many situations when inferring generic specialisations, and what mypy in particular does in lots of situations (though I don't think it's what mypy is doing here): bidirectional type inference. We would look later on in the same scope and see that an object of type Element[str] is required in order for the testsuite = ET.SubElement(testsuites, "testsuite") call to succeed without any typing errors, and use that information as part of our decision-making process when deciding what type to infer for the testsuites variable.

All of these options have their tradeoffs in terms of the user experience and their implementation complexities. It's a general issue that we're aware of and that we're still debating how to fix.

[Guitaricet]

Thank you for a quick reply! I think all these solutions sounds reasonable. It might be easier to switch from pytype to ty if they follow similar logic, but there are probably some issues with pytype's approach that I don't know about (and it's very slow).

I think I don't fully understand the advantage of the first approach, because we explicitly provide a "string literal" and I don't even know how to make it not literal. So, assuming that it's a union of Unknown|Literal is a bit dangerous, because the type is is fact known. Especially from the standpoint of the developer -- all code is very explicit with datatypes here.

Anyways, thanks for reacting. Just wanted to provide another potential test-case.

[JelleZijlstra]

I think this is also possibly a typeshed issue. SubElement is declared as taking just Element, which is Element[str] because of a TypeVar default, but I suspect it doesn't actually require a specific kind of Element.

(Side note: I feel we may have gone a bit overboard with TypeVar defaults. They can and do cause subtle usability issues. It's also sort of annoying that Element is invariant in its TypeVar; it can't be covariant because in theory somebody could be assigning to the tag field.)

Even if a typeshed fix helps in this specific case, you might run into similar issues elsewhere too. The Unknown | solution feels a bit dangerous because you're injecting (essentially) Any into what looks like fully typed code. Upcasting Literals in inferring generics feels a little unprincipled but might work well in practice.

[dcreager]

• We could apply a general principle that we should always upcast Literal types when inferring specialisations (I believe that is what both mypy and pyright are doing here); this would mean that we would infer Element[str] for the variable
• We could do something similar to what Rust does in many situations when inferring generic specialisations, and what mypy in particular does in lots of situations (though I don't think it's what mypy is doing here): bidirectional type inference. We would look later on in the same scope and see that an object of type Element[str] is required in order for the testsuite = ET.SubElement(testsuites, "testsuite") call to succeed without any typing errors, and use that information as part of our decision-making process when deciding what type to infer for the testsuites variable.

There's also an option in between these two, which you could consider a hacky approximation of bidirectional type checking: infer the literal type when inferring a specialization of a function, and infer the instance type when inferring a specialization of a class. The rationale being that a class specialization is more likely to be "persistent" or "long-lived", and therefore more likely that the widened instance type is what is intended. But that a function specialization is only (and can only be) used for a specific call site, and therefore more useful to propagate the more specific literal type into the return type.

(And to clarify, this would be a stopgap, not a long-term solution to this issue.)

[carljm]

I think the hybrid stopgap might lead to some unfortunate interactions in generic functions returning instances of generic classes?

class C[T]:
    x: T

def f[T](x: T) -> C[T]:
    return C(x)

f(1)

Is the last line an error because the generic function thinks it returns C[Literal[1]] but the class instantiation inside the function wants it to be C[int]?

[dcreager]

Is the last line an error because the generic function thinks it returns C[Literal[1]] but the class instantiation inside the function wants it to be C[int]?

I don't know that I'd call it an error — at least not in the sense of a diagnostic being produced. The revealed type for this example in astral-sh/ruff#18102 is specialized with a literal, though:

class C[T]:
    def __init__(self, x: T) -> None: ...

def f[T](x: T) -> C[T]:
    return C(x)

reveal_type(f(5))  # revealed: C[Literal[5]]

You might expect to get an invalid-return-type diagnostic, but that could only come from in the function body, and the literal type doesn't propagate from the call site into the function body.

Are you thinking that this might be an unsoundness?

[carljm]

The Unknown | solution feels a bit dangerous because you're injecting (essentially) Any into what looks like fully typed code.

A couple thoughts here:

1. The issue of "injecting Any into what looks like fully typed code" is kind of deeply baked into the typing spec for generics, in that the "default default" for any typevar is Any. So I'm not sure this is really all that new. You could see Element[str]("testsuites") as the "fully typed" version and Element("testsuites") as the "you didn't specify any upper bound" version, with implicit fallback, just like any other use of a not-explicitly-specialized generic type. (Although it seems like that fallback actually would be to Element[str] in this case, not Element[Any], due to the typevar default -- which means if we inferred Element("testsuites") as Element[upper-bound | lower-bound] it would actually become Element[str | Literal["testsuites"]) (not Element[Unknown | Literal["testsuites"]]), which of course simplifies to just Element[str]. Which is the desired behavior in this case.
2. I also think it's less dangerous than it seems, because of the preservation of the known lower bound (the | Literal["testsuites"] part of the type), which ensures that anytime you pull something out of the generic and try to use it in a way that would be an error for a string, you still get an error. I think that overall this approach gives a very good balance of avoiding assuming unspecified intent (and emitting false positives as a result), while still avoiding false negatives.
3. I think really the most challenging part of this idea is that if we e.g. infer list[Unknown | Literal[1]] for mylist = [1], we have to be able to flow-sensitively "narrow" its type to list[Unknown | Literal[1] | Literal["foo"]] after mylist.append("foo"), or the whole approach falls apart. This might turn out to be a deal-breaker (due to implementation complexity) in practice.

[dcreager]

I mentioned in astral-sh/ruff#18102 but not here that the literal promotion also does not happen when you explicitly specialize a generic class:

class C[T]:
    def __init__(self, x: T) -> None: ...


reveal_type(C(1))  # revealed: C[int]
reveal_type(C[int](1))  # revealed: C[int]
reveal_type(C[Literal[1]](1))  # revealed: C[Literal[1]]

If you squint this framing applies to the generic-function-returns-generic-class example, too — you've explicitly specialized the class in the return type annotation, and so my proposed literal promotion doesn't apply.

[carljm]

You might expect to get an invalid-return-type diagnostic, but that could only come from in the function body, and the literal type doesn't propagate from the call site into the function body.

Are you thinking that this might be an unsoundness?

No, you're right, I wasn't thinking clearly; this is a non-issue.