Incompatibles types in assignment

[remdragon]

• Are you reporting a bug, or opening a feature request?

This looks like a bug to me

• Please insert below the code you are checking with mypy,
  or a mock-up repro if the source is private. We would appreciate
  if you try to simplify your case to a minimal repro.

from __future__ import print_function

import builtins
from datetime import date, datetime, time, timedelta
from decimal import Decimal
from six import text_type as unicode # pip install six

if not getattr ( builtins, 'long', None ):
	long = int

MYPY = False
if MYPY:
	from typing import List, Optional, Text, Union
	VALUE = Union[int,long,float,Decimal,unicode,datetime,date,time,timedelta]

class Variant ( object ):
	name = None # type: Text
	value = None # type: Optional[VALUE]
	
	def __init__ ( self, name, value ):
		# type: ( Text, Optional[VALUE] ) -> None
		self.name = name
		self.value = value

def GetParams():
	# type: () -> List[Variant]
	return [ Variant ( u'Foo', u'Bar' ) ]

params = [] # type: List[Union[Variant,VALUE]]

params = GetParams() # THIS IS THE LINE THAT THROWS THE ERROR

for param in params:
	if isinstance ( param, Variant ):
		print ( param.value )
	else:
		print ( repr ( param ) )

• What is the actual behavior/output?

error: Incompatible types in assignment (expression has type "List[Variant]", variable has type "List[Union[Variant, int, int, float, Decimal, unicode, datetime, date, time, timedelta]]")

• What is the behavior/output you expect?

I would expect that I should be able to assign an expression of type List[Variant] to a variable of type List[Union[Variant,VALUE]]

• What are the versions of mypy and Python you are using?
  Do you see the same issue after installing mypy from Git master?

mypy 0.630
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 16:07:46) [MSC v.1900 32 bit (Intel)] on win32
Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:22:17) [MSC v.1500 32 bit (Intel)] on win32
I have not tried git master. I'm terribly sorry, but I unfortunately don't have the time to try to setup an environment where I can test with git master.

• What are the mypy flags you are using? (For example --strict-optional)

py2: --ignore-missing-imports --strict --py2
py3: --ignore-missing-imports --strict

[gvanrossum]

This is not a bug in mypy. Your issue is that List[Union[int, str]] is not a subtype of List[int]. To understand this, read up on Liskov substitution and variance: https://mypy.readthedocs.io/en/latest/generics.html#variance-of-generic-types

[ilevkivskyi]

Same misunderstanding appeared previously in #4186 (and in other issues). I am raising priority of that issue to high (also there is a simple solution, add the same note with the link to invariance that we add for bad argument type).

[remdragon]

gvanrossum / ilevkivskyi, thank you for your attention on this.

I researched the suggested topics worked up a couple more examples and at first I couldn't see the problem. I decided to document some of my own thought process on this in case it helps others.

In my example, the problem is that GetParams() could save a reference to the List it returns for later manipulation. It would assume (rightly) that the List could ever only contain Variant objects, but params has a Union and so could violate the type of the saved copy of the list inside GetParams().

Presumably then, the fix in my case would be to make a copy of the List returned, but this still fails (see code example below).

So I did even more research and now I think I understand the problem. While this code is technically correct in it's current state, it fails the "Liskov substitution code smell test" and that boils down to "there's probably a cleaner way to implement this".

Right after I posted this issue, in the interest of moving past it, I did end up finding a cleaner solution which essentially involved params below no longer needing to expect to have to contain a VALUE and so the type definition of params became List[Variant] and mypy is now happy and I now don't have to do instanceof checks on it's values.

tldr; you guys are awesome, thanks!!!!

from __future__ import print_function

import builtins
from datetime import date, datetime, time, timedelta
from decimal import Decimal
from six import text_type as unicode # pip install six

if not getattr ( builtins, 'long', None ):
	long = int

MYPY = False
if MYPY:
	from typing import List, Optional, Text, Union
	VALUE = Union[int,long,float,Decimal,unicode,datetime,date,time,timedelta]

class Variant ( object ):
	name = None # type: Text
	value = None # type: Optional[VALUE]
	
	def __init__ ( self, name, value ):
		# type: ( Text, Optional[VALUE] ) -> None
		self.name = name
		self.value = value

def GetParams():
	# type: () -> List[Variant]
	return [ Variant ( u'Foo', u'Bar' ) ]

params = [] # type: List[Union[Variant,VALUE]]

params = GetParams()[:] # make a copy but this still fails

for param in params:
	if isinstance ( param, Variant ):
		print ( param.value )
	else:
		print ( repr ( param ) )

[ilevkivskyi]

Yes, mypy isn't smart enough to figure out that making a copy is safe. This can probably be solved by giving list.copy() and list.__getitem__() a better signature in typeshed in the same way as python/typeshed#2404 does. cc @JelleZijlstra