Mypy 0.910 gives an internal error on xarray.core.common.full_like

[haroldmeerwaldt]

What happened:
mypy raised an internal error when checking importing xarray:

C:\Users\xxx\env\lib\site-packages\xarray\core\common.py:1664: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.910
Traceback (most recent call last):
  File "C:\Python397\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python397\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "mypy\checkexpr.py", line 3911, in accept
  File "mypy\nodes.py", line 1558, in accept
  File "mypy\checkexpr.py", line 271, in visit_call_expr
  File "mypy\checkexpr.py", line 353, in visit_call_expr_inner
  File "mypy\checkexpr.py", line 858, in check_call_expr_with_callee_type
  File "mypy\checkexpr.py", line 917, in check_call
  File "mypy\checkexpr.py", line 1014, in check_callable_call
  File "mypy\checkexpr.py", line 1485, in check_argument_types
  File "mypy\checkexpr.py", line 1515, in check_arg
  File "mypy\subtypes.py", line 93, in is_subtype
  File "mypy\subtypes.py", line 147, in _is_subtype
  File "mypy\types.py", line 1790, in accept
  File "mypy\subtypes.py", line 476, in visit_union_type
  File "mypy\subtypes.py", line 205, in _is_subtype
  File "mypy\subtypes.py", line 93, in is_subtype
  File "mypy\subtypes.py", line 147, in _is_subtype
  File "mypy\types.py", line 290, in accept
  File "mypy\subtypes.py", line 479, in visit_type_guard_type
RuntimeError: TypeGuard should not appear here
C:\Users\xxx\env\lib\site-packages\xarray\core\common.py:1664: : note: use --pdb to drop into pdb

It appears to have to do with this piece of code on line 1664 in xarray.core.common.full_like:

    if isinstance(other, Dataset):
        if not isinstance(fill_value, dict):
            fill_value = {k: fill_value for k in other.data_vars.keys()}

What you expected to happen:
I expected mypy to pass without errors. In 0.18.2 it still worked:

pip install "xarray==0.18.2"
pip install mypy
mypy -c "import xarray" --show-traceback
>>> Success: no issues found in 1 source file

Minimal Complete Verifiable Example:

pip install xarray
pip install mypy
mypy -c "import xarray" --show-traceback

Anything else we need to know?:
mypy version: 0.910
xarray version: 0.20.1
xarray 0.18.2 works fine, but 0.19.0 does not

Environment:

Output of xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 78 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: ('Dutch_Netherlands', '1252')
libhdf5: 1.12.1
libnetcdf: None
xarray: 0.20.1
pandas: 1.3.4
numpy: 1.21.4
scipy: 1.7.2
netCDF4: None
pydap: None
h5netcdf: 0.11.0
h5py: 3.5.0
Nio: None
zarr: None
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: 3.4.3
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
setuptools: 58.5.3
pip: 21.3.1
conda: None
pytest: 6.2.5
IPython: 7.29.0
sphinx: None

[max-sixty]

Thanks for the issue @haroldmeerwaldt . Is this an issue with mypy though? FWIW I get a different error, which may be intentional:

❯ mypy --version
mypy 0.910

❯ mypy -c "import xarray" --show-traceback
usage: mypy [-h] [-v] [-V] [more options; see below]
            [-m MODULE] [-p PACKAGE] [-c PROGRAM_TEXT] [files ...]
mypy: error: May only specify one of: module/package, files, or command.

[haroldmeerwaldt]

Hi @max-sixty, yes you could definitely say this is an issue with mypy. I wanted to raise it here in case somebody using xarray encountered the same error.

In the meantime I tried the commands I posted on an entirely clean environment, and then I do not get the error. Should have done that before, apologies. Something besides xarray and mypy seems to be causing this in my full environment.

Could you leave this issue open for a bit while I investigate? Perhaps I can help somebody in the future encountering the same thing.

Interesting that mypy fails on the command itself... What happens when you do: mypy -c "import xarray" or mypy --show-traceback -c "import xarray"? It seems it doesn't like --show-traceback at the end in your case.

[max-sixty]

Seems like you weren't the only person who had an env issue! Mine now works too:

❯ mypy --show-traceback -c "import xarray"
Success: no issues found in 1 source file

We can leave it open if you prefer! It does seem like a mypy issue though. So when you're comfortable we can close it, and if anyone else encounters it then reconsider.

[haroldmeerwaldt]

After a bit of investigating, I found that to reproduce this error, you need the --check-untyped-defs option (either on the command line or in the mypy.ini file):

pip install xarray
pip install mypy
mypy --show-traceback --check-untyped-defs -c "import xarray" 

The issue was already posted at mypy: python/mypy#11007 and a fix was merged in the dev branch.

If you want to fix this, you can:

• use the development mypy build: https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
• remove the --check-untyped-defs option
• or wait for the development mypy build to be released.

[sunayana]

Would it be possible to know which version of mypy has this fix now?