Failing functional tests with Python 3.9 #3895

sbraz · 2020-10-12T00:28:55Z

Steps to reproduce

Attempt to run tests with Python 3.9 with the git master

Current behavior

Hi, currently 5 tests fail with Python 3.9:

FAILED tests/test_functional.py::test_functional[unused_typing_imports] - Failed: Wrong results for file "unused_typing_imports":
FAILED tests/test_functional.py::test_functional[too_few_public_methods_37] - Failed: Wrong results for file "too_few_public_methods_37":
FAILED tests/test_functional.py::test_functional[star_needs_assignment_target_py35] - Failed: Wrong results for file "star_needs_assignment_target_py35":
FAILED tests/test_functional.py::test_functional[missing_kwoa_py3] - Failed: Wrong results for file "missing_kwoa_py3":
FAILED tests/test_functional.py::test_functional[disable_msg_github_issue_1389] - Failed: Wrong results for file "disable_msg_github_issue_1389":

Additional info

With @JulienPalard's changes in #3890, there are only 3 failures left:

FAILED tests/test_functional.py::test_functional[disable_msg_github_issue_1389]
FAILED tests/test_functional.py::test_functional[too_few_public_methods_37]
FAILED tests/test_functional.py::test_functional[star_needs_assignment_target_py35]

=================================== FAILURES ===================================
________________ test_functional[disable_msg_github_issue_1389] ________________

test_file = FunctionalTest:disable_msg_github_issue_1389

    @pytest.mark.parametrize("test_file", TESTS, ids=TESTS_NAMES)
    def test_functional(test_file):
        LintTest = (
            LintModuleOutputUpdate(test_file)
            if UPDATE
            else testutils.LintModuleTest(test_file)
        )
        LintTest.setUp()
>       LintTest._runTest()

tests/test_functional.py:102: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pylint.testutils.LintModuleTest object at 0x7f55b4e37ca0>

    def _runTest(self):
        modules_to_check = [self._test_file.source]
        self._linter.check(modules_to_check)
        expected_messages, expected_text = self._get_expected()
        received_messages, received_text = self._get_received()
    
        if expected_messages != received_messages:
            msg = ['Wrong results for file "%s":' % (self._test_file.base)]
            missing, unexpected = multiset_difference(
                expected_messages, received_messages
            )
            if missing:
                msg.append("\nExpected in testdata:")
                msg.extend(" %3d: %s" % msg for msg in sorted(missing))
            if unexpected:
                msg.append("\nUnexpected in testdata:")
                msg.extend(" %3d: %s" % msg for msg in sorted(unexpected))
>           pytest.fail("\n".join(msg))
E           Failed: Wrong results for file "disable_msg_github_issue_1389":
E           
E           Unexpected in testdata:
E              9: inherit-non-class
E             14: inherit-non-class

pylint/testutils.py:610: Failed
__________________ test_functional[too_few_public_methods_37] __________________

test_file = FunctionalTest:too_few_public_methods_37

    @pytest.mark.parametrize("test_file", TESTS, ids=TESTS_NAMES)
    def test_functional(test_file):
        LintTest = (
            LintModuleOutputUpdate(test_file)
            if UPDATE
            else testutils.LintModuleTest(test_file)
        )
        LintTest.setUp()
>       LintTest._runTest()

tests/test_functional.py:102: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pylint.testutils.LintModuleTest object at 0x7f55b2355670>

    def _runTest(self):
        modules_to_check = [self._test_file.source]
        self._linter.check(modules_to_check)
        expected_messages, expected_text = self._get_expected()
        received_messages, received_text = self._get_received()
    
        if expected_messages != received_messages:
            msg = ['Wrong results for file "%s":' % (self._test_file.base)]
            missing, unexpected = multiset_difference(
                expected_messages, received_messages
            )
            if missing:
                msg.append("\nExpected in testdata:")
                msg.extend(" %3d: %s" % msg for msg in sorted(missing))
            if unexpected:
                msg.append("\nUnexpected in testdata:")
                msg.extend(" %3d: %s" % msg for msg in sorted(unexpected))
>           pytest.fail("\n".join(msg))
E           Failed: Wrong results for file "too_few_public_methods_37":
E           
E           Unexpected in testdata:
E             23: inherit-non-class
E             23: too-few-public-methods

pylint/testutils.py:610: Failed
______________ test_functional[star_needs_assignment_target_py35] ______________

test_file = FunctionalTest:star_needs_assignment_target_py35

    @pytest.mark.parametrize("test_file", TESTS, ids=TESTS_NAMES)
    def test_functional(test_file):
        LintTest = (
            LintModuleOutputUpdate(test_file)
            if UPDATE
            else testutils.LintModuleTest(test_file)
        )
        LintTest.setUp()
>       LintTest._runTest()

tests/test_functional.py:102: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pylint.testutils.LintModuleTest object at 0x7f55afcbadc0>

    def _runTest(self):
        modules_to_check = [self._test_file.source]
        self._linter.check(modules_to_check)
        expected_messages, expected_text = self._get_expected()
        received_messages, received_text = self._get_received()
    
        if expected_messages != received_messages:
            msg = ['Wrong results for file "%s":' % (self._test_file.base)]
            missing, unexpected = multiset_difference(
                expected_messages, received_messages
            )
            if missing:
                msg.append("\nExpected in testdata:")
                msg.extend(" %3d: %s" % msg for msg in sorted(missing))
            if unexpected:
                msg.append("\nUnexpected in testdata:")
                msg.extend(" %3d: %s" % msg for msg in sorted(unexpected))
>           pytest.fail("\n".join(msg))
E           Failed: Wrong results for file "star_needs_assignment_target_py35":
E           
E           Expected in testdata:
E             15: star-needs-assignment-target
E           
E           Unexpected in testdata:
E             15: syntax-error

pylint/testutils.py:610: Failed
=============================== warnings summary ===============================
../usr/local/lib/python3.9/site-packages/astroid/interpreter/_import/spec.py:15
  /usr/local/lib/python3.9/site-packages/astroid/interpreter/_import/spec.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

tests/test_functional.py::test_functional[future_unicode_literals]
tests/test_functional.py::test_functional[anomalous_unicode_escape_py3]
  <unknown>:5: DeprecationWarning: invalid escape sequence \u

tests/test_functional.py::test_functional[anomalous_unicode_escape_py3]
  <unknown>:6: DeprecationWarning: invalid escape sequence \U

tests/test_functional.py::test_functional[anomalous_unicode_escape_py3]
  <unknown>:8: DeprecationWarning: invalid escape sequence \N

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/test_functional.py::test_functional[disable_msg_github_issue_1389]
FAILED tests/test_functional.py::test_functional[too_few_public_methods_37]
FAILED tests/test_functional.py::test_functional[star_needs_assignment_target_py35]
============ 3 failed, 385 passed, 23 skipped, 5 warnings in 35.47s ============

Reproducer Dockerfile (I'm only running test_functionalbecause other tests pass):

FROM python:3.9
RUN git clone https://github.com/PyCQA/pylint --depth 1
RUN pip install astroid isort pytest
WORKDIR pylint
RUN git fetch origin pull/3890/head:pr3890 --depth 1 && git checkout pr3890
RUN PYTHONPATH=. pytest -vv tests/test_functional.py

The text was updated successfully, but these errors were encountered:

tirkarthi · 2020-10-12T14:35:54Z

disable_msg_github_issue_1389 seems to involve inheriting from NamedTuple which is covered in #3876

tirkarthi · 2020-10-12T15:01:02Z

For the syntax related issue it seems to be due to the new parser since using old parser parses the the statement though the statement is not legal.

$ python3.9                                                        
Python 3.9.0 (default, Oct  6 2020, 02:47:17) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.parse("UNPACK_IN_COMP = {elem for elem in (*range(10))}")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.9/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
  File "<unknown>", line 1
    UNPACK_IN_COMP = {elem for elem in (*range(10))}
                                        ^
SyntaxError: can't use starred expression here

$ python3.9 -X oldparser                                           
Python 3.9.0 (default, Oct  6 2020, 02:47:17) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.parse("UNPACK_IN_COMP = {elem for elem in (*range(10))}")
<ast.Module object at 0x7f469469dd30>

tirkarthi · 2020-10-12T15:02:40Z

The syntax issue seems to have been closed in https://bugs.python.org/issue40631

tirkarthi · 2020-10-12T15:09:22Z

too_few_public_methods_37 also seems to involve inheriting from NamedTuple which is covered in #3876

sbraz · 2020-10-12T15:17:03Z

Thanks for the analysis, this means there is only one actual issue with NamedTuple, right? What should be done about the star syntax that was not valid anyway? Skip it?

tirkarthi · 2020-10-12T15:21:35Z

yes, something like below could be added to skip syntax related issue :

diff --git a/tests/functional/s/star_needs_assignment_target_py35.rc b/tests/functional/s/star_needs_assignment_target_py35.rc
index 71de8b6..b743e8f 100644
--- a/tests/functional/s/star_needs_assignment_target_py35.rc
+++ b/tests/functional/s/star_needs_assignment_target_py35.rc
@@ -1,2 +1,3 @@
 [testoptions]
 min_pyver=3.5
+max_pyver=3.8

hippo91 · 2020-10-17T10:14:40Z

Thanks all for your very interesting messages. We definitely need to fix this and support python 3.9.

JulienPalard · 2020-10-18T19:56:04Z

@hippo91 There's #3905 and #3890 that may help.

hippo91 · 2020-10-23T10:14:01Z

@JulienPalard thanks for your message. I'll try to make a review ASAP.

hippo91 · 2020-10-31T16:23:58Z

I'm working on those problems. In fact the NameTuple problem is dual. Solving the inherit-non-class false positive doesn't implicate solving too-few-public-methods. I got quite a proper fix for the first one but sadly the other one is much more complex and deal with astroid and inference.
For python versions prior to 3.9, NameTuple was a class of the typing module. With python3.9 it became a kind of factory which results is much more difficult to infer. I'll try to find a proper solution ASAP.
@tirkarthi it seems the bug you mention was in fact a false positive. I think we will have to solve the syntax error too.

hippo91 added Blocker 🙅 Blocks the next release Bug 🪲 python 3.9 labels Oct 17, 2020

cdce8p mentioned this issue Oct 18, 2020

Fix for py39 - unsubscriptable-object #3905

Closed

4 tasks

hippo91 self-assigned this Oct 31, 2020

hippo91 added the Work in progress label Oct 31, 2020

This was referenced Nov 24, 2020

inherit-non-class false positive: NamedTuple (Python 3.9) #3876

Closed

Fix python39 pylint-dev/astroid#858

Merged

Toward python39 #3959

Merged

hippo91 closed this as completed in #3959 Dec 29, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing functional tests with Python 3.9 #3895

Failing functional tests with Python 3.9 #3895

sbraz commented Oct 12, 2020 •

edited

Loading

tirkarthi commented Oct 12, 2020

tirkarthi commented Oct 12, 2020

tirkarthi commented Oct 12, 2020

tirkarthi commented Oct 12, 2020

sbraz commented Oct 12, 2020

tirkarthi commented Oct 12, 2020

hippo91 commented Oct 17, 2020

JulienPalard commented Oct 18, 2020

hippo91 commented Oct 23, 2020

hippo91 commented Oct 31, 2020

Failing functional tests with Python 3.9 #3895

Failing functional tests with Python 3.9 #3895

Comments

sbraz commented Oct 12, 2020 • edited Loading

Steps to reproduce

Current behavior

Additional info

tirkarthi commented Oct 12, 2020

tirkarthi commented Oct 12, 2020

tirkarthi commented Oct 12, 2020

tirkarthi commented Oct 12, 2020

sbraz commented Oct 12, 2020

tirkarthi commented Oct 12, 2020

hippo91 commented Oct 17, 2020

JulienPalard commented Oct 18, 2020

hippo91 commented Oct 23, 2020

hippo91 commented Oct 31, 2020

sbraz commented Oct 12, 2020 •

edited

Loading