TypeGuard not working as I expected

[ghlecl]

Hello,

I am taking our code base and adding type hints to it. In the course of doing this, I have created functions that are TypeGuards to help with the type hints. As mentioned in the title, some of the TypeGuards do not seem to work as I would expect (which is not the same as not working as anybody else would expect: I am trying to convey my expectations might be completely wrong.

Although the details are probably not relevant, some idea of the design might be relevant. I have a class NameInterface, from which a series of other classes inherit (GtvNm, PtvNm, UGtvNm, etc., roughly a dozen subclasses):

class NameInterface( object ):
   pass

class GtvNm( NameInterface, LabelMixin ):
   pass

A function name_from_str analyses a string and returns an object of the appropriate type:

def name_from_str( strg: str ) -> NameInterface:
   # code goes here

Then, I have defined a bunch of type aliases and TypeGuard functions to help type hinting when working with the object. I have done it like so:

GenericSubTarget = Union[UGtvNm, UCtvNm, UItvNm]
GenericTarget = Union[GenericSubTarget,UPtvNm]
SubTarget = Union[GtvNm, CtvNm, ItvNm]
Target = [Union[SubTarget],PtvNm]

def is_generic_target( nm: NameInterface ) -> TypeGuard[GenericTarget]:
   # code to check with branches for True and False
   # a GenericTarget is guaranteed to have a .level member, for instance
   return True

Unfortunately, when used in code, the function as written above does not seems to not act as a type guard. For instance:

for comp in nm.components:
   if is_generic_target( comp ) and ( comp.level != GenericTargetLevel.UNSPECIFIED() ):
      cur_spec_name_pairs = self.expand_generic_name( comp )

will "complain" (i.e. put squiggles under) the .level function/property call. I would have expected that the function acting as a TypeGuard would then understand that comp is a GenericTarget and would see/consider that it is guaranteed to have a .level member. I have tried putting both checks on different lines (thinking maybe it is because of the and), but it makes no difference.

Are my expectations wrong ? Are there restrictions on TypeGuards I am not aware of (read the python documentation, quite a few forums, tried to find the info on my own). ? Is this a bug ? Probably a bug, but in my code or in the type checker ? Might it be that what I am doing is not actually type narrowing in some sense ?

I have a screen shot of the squigglies in Sublime Text using LSP-Pyright (not that it will help much, but to show what I mean).

Any insight or help will be welcome. Sorry for being verbose.

[erictraut]

Could you provide a single, self-contained, minimal code sample that demonstrates your issue? Your code above is piecemeal and refers to symbols that are not defined. I could try to guess at the definition of these symbols, but I'd appreciate it if you could provide a single, complete sample.

[ghlecl]

I will try to write one right away and get back to you if I can reproduce my bug in a "simpler" hierarchy.
I do not have the right to publish the full code and anyhow, it would be too big unnecessarily.
If I cannot reproduce, I will also write it here and/or try and delete my question.
Thanks in any case.

[ghlecl]

#
# stuck in python 3.8 for reasons that are out of our hands.
#
from typing_extensions import Protocol, TypeGuard
from typing import Optional, Union, List


#-----------------------------------------------------------------------------------------
class NameLabel( object ):
   @staticmethod
   def from_str( strg: str ) -> "NameLabel":
      return NameLabel( txt = strg )

   def __init__( self, txt, margin = None ):
      self.txt = txt
      self.margin = margin


#-----------------------------------------------------------------------------------------
class SupportLabel( Protocol ):
   @property
   def label( self ) -> Optional[NameLabel]: ...

   @label.setter
   def label( self, label: Optional[Union[str,NameLabel]] ): ...


#-----------------------------------------------------------------------------------------
class LabelMixin( SupportLabel ):
   @property
   def label( self ) -> Optional[NameLabel]:
      return self.__label

   @label.setter
   def label( self, lbl: Optional[Union[str,NameLabel]] ):
      if lbl is not None:
         if isinstance( lbl, str ): lbl = NameLabel.from_str( lbl )
         if not isinstance( lbl, NameLabel ): raise ValueError( "invalid label" )
      self.__label = lbl


#-----------------------------------------------------------------------------------------
class LevelMixin( object ):
   @property
   def level( self ) -> Optional[int]:
      return self.__level

   @level.setter
   def level( self, lvl: Optional[int] ):
      if lvl is not None:
         if not isinstance( lvl, int ): raise ValueError( "invalid level" )
         if lvl < 0: raise ValueError( "invalid level" )
      self.__level = lvl


#-----------------------------------------------------------------------------------------
class NameInterface( object ):
   pass


#-----------------------------------------------------------------------------------------
class UGtvCtvItvNmBase( NameInterface, LabelMixin, LevelMixin ):
   _is_generic = True

   def __init__( self, level: Optional[int], label: Optional[NameLabel] = None ):
      self.level = level
      self.label = label

#-----------------------------------------------------------------------------------------
class UGtvNm( UGtvCtvItvNmBase ):
   def __str__( self ) -> str:
      return f"GTV{self.level if self.level is not None else ''}"


#-----------------------------------------------------------------------------------------
class UCtvNm( UGtvCtvItvNmBase ):
   def __str__( self ) -> str:
      return f"CTV{self.level if self.level is not None else ''}"


#-----------------------------------------------------------------------------------------
class UPtvNm( NameInterface, LabelMixin, LevelMixin ):
   def __init__( self, trt_grp: int, level: Optional[int], label: Optional[NameLabel] = None ):
      self.level = level
      self.label = label
      self.trt_grp = trt_grp


   def __str__( self ) -> str:
      return f"{self.trt_grp}PTV{self.level if self.level is not None else ''}"



#-----------------------------------------------------------------------------------------
class CompositeNm( NameInterface ):
   def __init__( self, components: List[NameInterface] ):
      self._components = components

   @property
   def components( self ) -> List[NameInterface]:
      return self._components



#-----------------------------------------------------------------------------------------
SubTargets = Union[UGtvNm, UCtvNm]
Targets = Union[SubTargets,UPtvNm]

def is_subtarget( nm: NameInterface ) -> TypeGuard[SubTargets]:
   return isinstance( nm, ( UGtvNm, UCtvNm ) )

def is_target( nm: NameInterface ) -> TypeGuard[Targets]:
   return isinstance( nm, ( UGtvNm, UCtvNm, UPtvNm ) )

def is_generic( nm: NameInterface ) -> bool:
   return getattr( nm, "_is_generic", False )

def is_generic_subtarget( nm: NameInterface ) -> TypeGuard[SubTargets]:
   return is_target( nm ) and is_generic( nm )



#-----------------------------------------------------------------------------------------
class SomeClassInOtherProject( object ):
   def some_fn( self, nm: NameInterface ):
      if isinstance( nm, CompositeNm ):
         for comp in nm.components:
            if is_generic_subtarget( comp ) and comp.level != 2:
               print( "here" )


nm = CompositeNm( [ UPtvNm( 1, 1 ), UGtvNm( 1 ) ] )

obj = SomeClassInOtherProject()
obj.some_fn( nm )

This is the design we have (or very close to it).

Not sure if it will help the discussion to see it though (but just in case, I decided to post it here): I cannot reproduce the error with this simpler case, so maybe the problem is not related to TypeGuard and pyright, but to some other error we have somewhere. I will keep looking into it.

I don't know if it is better to keep this discussion thread open in case I can reproduce the problem in the next few days and report back, or if it is better to close it and for me to reopen a new one if I nail down the problem.

In any case, thanks for your time.

[xorcus]

TypeGuard not working as I expected too. Here's a basic TypeGuard example and it does not seem to work:

# Pylance v2024.11.3
# Pyright v1.1.390

# Return True if the supplied value is None, empty, or contains only whitespace.
def is_blank(value: str | None) -> TypeGuard[None]:
    return not value or value.isspace()

value: str | None = os.getenv("MY_VAR")
if (is_blank(value)):
    value = "not None"

strVal: str = value  # not expecting warning: "None" is not assignable to "str" (PylancereportAssignmentType)

[erictraut]

@xorcus, TypeGuard does not perform narrowing in the negative ("else") case. Your code example relies on negative narrowing. If you switch from TypeGuard to TypeIs, your code will work as you expect.

[xorcus]

@erictraut, thanks for your reply. I tried, but couldn't find where to import TypeIs from.
Tried from typing import TypeIs, it complains as follows. Do I need to upgrade? In which version was TypeIs introduced?

"TypeIs" is unknown import symbol - Pylance reportAttributeAccessIssue
Type of "TypeIs" is unknown - Pylance reportUnknownVariableType
(import) TypeIs: Unknown

[xorcus]

ah, found it by browsing this project's sources:
from typing_extensions import TypeIs

[erictraut]

TypeIs was added to the typing module in Python 3.13 (PEP 742). For older versions of Python, you can import it from typing_extensions.