Crash with latest mypy and django-stubs #1412

matejsp · 2023-03-23T09:46:40Z

Bug report

Error running latest django-stubs.

Traceback (most recent call last):
  File "mypy/checkexpr.py", line 4881, in accept
  File "mypy/nodes.py", line 1881, in accept
  File "mypy/checkexpr.py", line 429, in visit_call_expr
  File "mypy/checkexpr.py", line 549, in visit_call_expr_inner
  File "mypy/checkexpr.py", line 1209, in check_call_expr_with_callee_type
  File "mypy/checkexpr.py", line 1292, in check_call
  File "mypy/checkexpr.py", line 1502, in check_callable_call
  File "mypy/checkexpr.py", line 1049, in apply_function_plugin
  File "/Users/myuser/.virtualenvs/py311/lib/python3.11/site-packages/mypy_django_plugin/transformers/querysets.py", line 233, in extract_proper_type_queryset_annotate
    annotated_type = get_or_create_annotated_type(api, model_type, fields_dict=fields_dict)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/myuser/.virtualenvs/py311/lib/python3.11/site-packages/mypy_django_plugin/transformers/models.py", line 669, in get_or_create_annotated_type
    model_module_file = api.modules[model_module_name]  # type: ignore
                        ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'django_stubs_ext'

It works using django-stubs==1.7.0 without django-stubs-ext installed.

What's wrong

Got crash with django-stubs.

How is that should be

It should not crash.

System information

OS: MacOSX Ventura 12.2
python version: 3.11.2 or 3.8.13
django version: 3.2.18
mypy version: 1.1.1
django-stubs version: 1.16.0
django-stubs-ext version: 0.8.0

The text was updated successfully, but these errors were encountered:

matejsp · 2023-03-23T09:48:02Z

[mypy]
warn_return_any = True
warn_unused_configs = True
disallow_untyped_defs = False
ignore_missing_imports = True
# follow_imports = normal
follow_imports = skip
files = api,account,apps,common
plugins = mypy_django_plugin.main

[mypy.plugins.django-stubs]
DJANGO_SETTINGS_MODULE = settings

[mypy-common/user_history.*]
disallow_untyped_defs = True
# To allow introduction per bounded context
follow_imports = normal

matejsp · 2023-03-23T09:57:11Z

Also tested with:
django-stubs==1.8.0 -> WORKS
django-stubs==1.9.0 -> not working

matejsp · 2023-05-16T04:51:34Z

I upgraded to django-stubs==4.2.0 and django-stubs-ext==4.2.0 and it is still completely broken:

Traceback (most recent call last):
  File "mypy/checkexpr.py", line 4881, in accept
  File "mypy/nodes.py", line 1881, in accept
  File "mypy/checkexpr.py", line 429, in visit_call_expr
  File "mypy/checkexpr.py", line 549, in visit_call_expr_inner
  File "mypy/checkexpr.py", line 1209, in check_call_expr_with_callee_type
  File "mypy/checkexpr.py", line 1292, in check_call
  File "mypy/checkexpr.py", line 1502, in check_callable_call
  File "mypy/checkexpr.py", line 1049, in apply_function_plugin
  File "/Users/myuser/.virtualenvs/env311/lib/python3.11/site-packages/mypy_django_plugin/transformers/querysets.py", line 236, in extract_proper_type_queryset_annotate
    annotated_type = get_or_create_annotated_type(api, model_type, fields_dict=fields_dict)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matej.spiller-muys/.virtualenvs/bitstamp311/lib/python3.11/site-packages/mypy_django_plugin/transformers/models.py", line 669, in get_or_create_annotated_type
    model_module_file = api.modules[model_module_name]  # type: ignore
                        ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'django_stubs_ext'

Is there really no workaround for this problem?

sobolevn · 2023-05-16T05:52:49Z

Please, provide a reproducer. It clearly works for others :)

matejsp · 2023-05-16T06:20:22Z

Hm hard to reproduce but it crashes at line:
qry = models.DocumentRevision.objects.annotate(

    obsolete_at_filter_expression = (Q(obsolete_at__gte=datetime.datetime.now()) | Q(obsolete_at__isnull=True),)  # type: ignore  # noqa: E501
    if 'show_obsolete' in filtering and filtering['show_obsolete']:
        obsolete_at_filter_expression = tuple()  # type: ignore
        del filtering['show_obsolete']
    qry = models.DocumentRevision.objects.annotate(
        max_modified_date=Max('document__documentcontent__modified_at')
    ).filter(*obsolete_at_filter_expression, **filtering)

matejsp · 2023-05-16T06:27:39Z

And another crash after I comment annotate in previous one:

        data = (
            models.SomeEvent.objects.values('exception_name').order_by().annotate(Count('exception_name'))
        )

Traceback (most recent call last):
  File "mypy/checkexpr.py", line 4881, in accept
  File "mypy/nodes.py", line 1881, in accept
  File "mypy/checkexpr.py", line 429, in visit_call_expr
  File "mypy/checkexpr.py", line 549, in visit_call_expr_inner
  File "mypy/checkexpr.py", line 1209, in check_call_expr_with_callee_type
  File "mypy/checkexpr.py", line 1292, in check_call
  File "mypy/checkexpr.py", line 1502, in check_callable_call
  File "mypy/checkexpr.py", line 1049, in apply_function_plugin
  File "/Users/myuser/.virtualenvs/py311/lib/python3.11/site-packages/mypy_django_plugin/transformers/querysets.py", line 277, in extract_proper_type_queryset_values
    assert isinstance(default_return_type, Instance)
AssertionError:

matejsp · 2023-05-16T07:14:19Z

We use 4.1.9 Django now (different as in original bug, but same bug).

The key that is searched for type info couple of lines before in transfomers/models.py:

    annotated_typeinfo = helpers.lookup_fully_qualified_typeinfo(
        cast(TypeChecker, api), model_module_name + "." + type_name
    )

model_module_name + "." + type_name has a value:

"django_stubs_ext.WithAnnotations[models__DocumentRevision, TypedDict({'max_modified_date': Any})]"

And for some reason django_stubs_ext is missing in the list.

realsuayip · 2023-06-28T12:50:29Z

I can also confirm on Python 3.11.4 with

django-stubs==4.2.2
django-stubs-ext==4.2.2
mypy==1.4.1
django==4.2.2

If you disable mypy cache, it does not crash.

Here is a piece of offending code:

def get_relations(self, obj: WithAnnotations[User]) -> list[str]:
    return [name for name, exists in obj.rels.items() if exists]

matejsp · 2023-07-06T06:27:30Z

How did you disable mypy cache youself?
If I disable it using official docs it still does not work.

django-stubs==4.2.3
django-stubs-ext==4.2.2
mypy==1.4.1
django==4.1.9

version: 1.4.1
Traceback (most recent call last):
  File "mypy/checkexpr.py", line 4961, in accept
  File "mypy/nodes.py", line 1901, in accept
  File "mypy/checkexpr.py", line 428, in visit_call_expr
  File "mypy/checkexpr.py", line 548, in visit_call_expr_inner
  File "mypy/checkexpr.py", line 1208, in check_call_expr_with_callee_type
  File "mypy/checkexpr.py", line 1291, in check_call
  File "mypy/checkexpr.py", line 1501, in check_callable_call
  File "mypy/checkexpr.py", line 1048, in apply_function_plugin
  File "/Users/myuser/.virtualenvs/venv311/lib/python3.11/site-packages/mypy_django_plugin/transformers/querysets.py", line 233, in extract_proper_type_queryset_annotate
    annotated_type = get_or_create_annotated_type(api, model_type, fields_dict=fields_dict)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/myuser/.virtualenvs/venv311/lib/python3.11/site-packages/mypy_django_plugin/transformers/models.py", line 666, in get_or_create_annotated_type
    model_module_file = api.modules[model_module_name]  # type: ignore
                        ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'django_stubs_ext'
documents/services/document_revisions.py:329: : note: use --pdb to drop into pdb
(venv311) ➜  myuser mypy --cache-dir=/dev/null --show-traceback

realsuayip · 2023-07-06T06:48:10Z

I used --no-incremental

https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-no-incremental

Removing .mypy_cache manually and running again also works.

matejsp · 2023-07-06T07:01:05Z

Thank you for the fast answer. It seems my case is a bit different for some reason because it does not work in my case:
mypy --cache-dir=/dev/null --show-traceback --no-incremental

version: 1.4.1
Traceback (most recent call last):
  File "mypy/checkexpr.py", line 4961, in accept
  File "mypy/nodes.py", line 1901, in accept
  File "mypy/checkexpr.py", line 428, in visit_call_expr
  File "mypy/checkexpr.py", line 548, in visit_call_expr_inner
  File "mypy/checkexpr.py", line 1208, in check_call_expr_with_callee_type
  File "mypy/checkexpr.py", line 1291, in check_call
  File "mypy/checkexpr.py", line 1501, in check_callable_call
  File "mypy/checkexpr.py", line 1048, in apply_function_plugin
  File "/Users/myuser/.virtualenvs/venv311/lib/python3.11/site-packages/mypy_django_plugin/transformers/querysets.py", line 233, in extract_proper_type_queryset_annotate
    annotated_type = get_or_create_annotated_type(api, model_type, fields_dict=fields_dict)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/myuser/.virtualenvs/venv311/lib/python3.11/site-packages/mypy_django_plugin/transformers/models.py", line 666, in get_or_create_annotated_type
    model_module_file = api.modules[model_module_name]  # type: ignore
                        ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'django_stubs_ext'

Refs #1412

matejsp · 2023-07-06T19:28:08Z

I have managed to reproduce the issue with the following sample:
bug_report3.tar.gz

Inside is runme script

#!/bin/bash
set -eu -o pipefail

rm -rf .venv
rm -rf .mypy_cache

python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
mypy sample --show-traceback

I get:

(.venv) ➜  bug_report mypy sample
sample/document_revision.py:13: error: 'Settings' object has no attribute 'READ_ONLY_DB'  [misc]
sample/document_revision.py:19: error: 'Settings' object has no attribute 'READ_ONLY_DB'  [misc]
sample/document_revision.py:19: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
If this issue continues with mypy master, please report a bug at https://github.com/python/mypy/issues
version: 1.4.1
sample/document_revision.py:19: : note: please use --show-traceback to print a traceback when reporting a bug

First issue is with following imports #1614 and second is this one.

matejsp · 2023-07-06T19:35:06Z

@sobolevn I tried the patch with the reproducable in previous comment, but it does not work:(https://github.com/typeddjango/django-stubs/pull/1613/files#diff-e548f38fb43163048bb2a1c1a209654cbee3c97d5ca25cfe16ca0abc5065e7c6).

It seems to go:
(.venv) ➜ bug_report mypy sample --show-traceback
sample.document_revision -> added print(print(file.fullname))
not defined_model_classes -> print('not defined_model_classes')

Then I tried adding:

        defined_model_classes = self.django_context.model_modules.get(file.fullname)
        if not defined_model_classes:
            print('not defined_model_classes')
            return [self._new_dependency("django_stubs_ext")]

But still didn't work with the same error :(

If I change:
[mypy]
follow_imports = normal (instead of skip)

it starts working ...

Refs #1412

matejsp · 2023-09-04T06:49:06Z

For me current workaround is to add to mypy.ini:

[mypy]
follow_imports = skip

[mypy-django_stubs_ext.*]
follow_imports = normal

…ommand This PR implements the workaround shown in typeddjango/django-stubs#1412 (comment)

This PR implements the workaround shown in typeddjango/django-stubs#1412 (comment)

flaeppe · 2024-08-09T18:29:22Z

#1727 should've removed 1 error, according to the traceback. And the crashing code was removed in #2319.

Due to that I'm closing this. Feel free to post a new issue with any errors/tracebacks you're encountering.

matejsp added the bug Something isn't working label Mar 23, 2023

sobolevn added a commit that referenced this issue Jul 6, 2023

Try solving missing django-stubs-ext module

cb9c85c

Refs #1412

sobolevn mentioned this issue Jul 6, 2023

Try solving missing django-stubs-ext module #1613

Merged

sobolevn added a commit that referenced this issue Jul 9, 2023

Try solving missing django-stubs-ext module (#1613)

640f2c7

Refs #1412

flaeppe mentioned this issue Sep 21, 2023

Handle mismatching types in queryset method resolving gracefully #1727

Merged

ERosendo added a commit to freelawproject/bigcases2 that referenced this issue Jan 18, 2024

fix(mypy): Update the .ini file to avoid crashing while running the c…

b01630c

…ommand This PR implements the workaround shown in typeddjango/django-stubs#1412 (comment)

ERosendo added a commit to freelawproject/bigcases2 that referenced this issue Jan 18, 2024

fix(mypy): Update the .ini file to avoid crashing

922c828

This PR implements the workaround shown in typeddjango/django-stubs#1412 (comment)

ERosendo added a commit to freelawproject/bigcases2 that referenced this issue Jan 19, 2024

fix(mypy): Update the .ini file to avoid crashing

fc6ec13

This PR implements the workaround shown in typeddjango/django-stubs#1412 (comment)

ERosendo added a commit to freelawproject/bigcases2 that referenced this issue Jan 19, 2024

fix(mypy): Update the .ini file to avoid crashing

dd625f2

This PR implements the workaround shown in typeddjango/django-stubs#1412 (comment)

flaeppe closed this as completed Aug 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Crash with latest mypy and django-stubs #1412

Crash with latest mypy and django-stubs #1412

matejsp commented Mar 23, 2023

matejsp commented Mar 23, 2023

matejsp commented Mar 23, 2023

matejsp commented May 16, 2023

sobolevn commented May 16, 2023

matejsp commented May 16, 2023

matejsp commented May 16, 2023

matejsp commented May 16, 2023 •

edited

Loading

realsuayip commented Jun 28, 2023

matejsp commented Jul 6, 2023

realsuayip commented Jul 6, 2023

matejsp commented Jul 6, 2023 •

edited

Loading

matejsp commented Jul 6, 2023

matejsp commented Jul 6, 2023 •

edited

Loading

matejsp commented Sep 4, 2023

flaeppe commented Aug 9, 2024

Crash with latest mypy and django-stubs #1412

Crash with latest mypy and django-stubs #1412

Comments

matejsp commented Mar 23, 2023

Bug report

What's wrong

How is that should be

System information

matejsp commented Mar 23, 2023

matejsp commented Mar 23, 2023

matejsp commented May 16, 2023

sobolevn commented May 16, 2023

matejsp commented May 16, 2023

matejsp commented May 16, 2023

matejsp commented May 16, 2023 • edited Loading

realsuayip commented Jun 28, 2023

matejsp commented Jul 6, 2023

realsuayip commented Jul 6, 2023

matejsp commented Jul 6, 2023 • edited Loading

matejsp commented Jul 6, 2023

matejsp commented Jul 6, 2023 • edited Loading

matejsp commented Sep 4, 2023

flaeppe commented Aug 9, 2024

matejsp commented May 16, 2023 •

edited

Loading

matejsp commented Jul 6, 2023 •

edited

Loading

matejsp commented Jul 6, 2023 •

edited

Loading