Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code objects with non-equal co_linetables compare equal #95150

Closed
15r10nk opened this issue Jul 22, 2022 · 3 comments
Closed

Code objects with non-equal co_linetables compare equal #95150

15r10nk opened this issue Jul 22, 2022 · 3 comments
Labels
3.11 only security fixes 3.12 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) release-blocker triaged The issue has been accepted as valid by a triager. type-bug An unexpected behavior, bug, or error

Comments

@15r10nk
Copy link
Contributor

15r10nk commented Jul 22, 2022

Bug report

def f(a, b):
    return dict((k.lower(), v) for k, v in a.items()) == dict(
        (k.lower(), v) for k, v in b.items()
    )

f({"a": 5}, {1: 1})

output:

Traceback (most recent call last):
  File "/home/frank/projects/executing/test8.py", line 8, in <module>
    f({"a": 5}, {1: 1})
  File "/home/frank/projects/executing/test8.py", line 2, in f
    return dict((k.lower(), v) for k, v in a.items()) == dict(
                                                         ^^^^^
  File "/home/frank/projects/executing/test8.py", line 2, in <genexpr>
    return dict((k.lower(), v) for k, v in a.items()) == dict(
                 ^^^^^^^
AttributeError: 'int' object has no attribute 'lower'

The error should be reported at line 3 and not at line 2, because the integer is passed to b

Your environment

  • current cpython 3.11 branch (3a33e9b)
  • and the Python 3.11.0b4
@15r10nk 15r10nk added the type-bug An unexpected behavior, bug, or error label Jul 22, 2022
15r10nk added a commit to 15r10nk/executing that referenced this issue Jul 22, 2022
@alexmojaki
Copy link

Simpler reproduction:

[k.lower() for k in "a"], [k.lower() for k in [1]]
Traceback (most recent call last):
  File "...", line 1, in <module>
    [k.lower() for k in "a"], [k.lower() for k in [1]]
                              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "...", line 1, in <listcomp>
    [k.lower() for k in "a"], [k.lower() for k in [1]]
     ^^^^^^^^^
AttributeError: 'int' object has no attribute 'lower'

@brandtbucher
Copy link
Member

brandtbucher commented Jul 22, 2022

Ooh, nice find! The code objects are being "deduplicated" in the compiler, even though they have different column positions.

>>> def f():
...     (a for b in c), (a for b in d)
... 
>>> f.__code__.co_consts
(None, <code object <genexpr> at 0x7fa4fbf92640, file "<stdin>", line 2>)

We need to consider the entire positions table when comparing code objects for equality.

@brandtbucher brandtbucher added interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.11 only security fixes 3.12 bugs and security fixes triaged The issue has been accepted as valid by a triager. labels Jul 22, 2022
@brandtbucher brandtbucher changed the title wrong error position in traceback when using generator expressions Code objects with non-equal co_linetables compare equal Jul 23, 2022
@brandtbucher
Copy link
Member

@markshannon: we should add co_linetable to the code equality (and maybe hash?) functions.

Probably co_exceptiontable too, since I’m pretty sure it’s possible to have two code objects that differ only in exception table when compiling custom ASTs. We definitely don't want those to dedup.

miss-islington pushed a commit to miss-islington/cpython that referenced this issue Aug 1, 2022
…d equality (pythonGH-95509)

(cherry picked from commit c7e5bba)

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
miss-islington added a commit that referenced this issue Aug 1, 2022
…lity (GH-95509)

(cherry picked from commit c7e5bba)

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
Repository owner moved this from Todo to Done in Release and Deferred blockers 🚫 Aug 1, 2022
clrpackages pushed a commit to clearlinux-pkgs/pypi-executing that referenced this issue Sep 27, 2022
…rsion 1.1.0

Alex Hall (4):
      GHA: test 3.11 and PRs
      Use `only` instead of `find_node` function
      Ignore EXTENDED_ARG/NOP when counting CALL_FUNCTION ops
      3.11 in setup.cfg

Frank Hoffmann (91):
      feat: support for python 3.11
      fix: fixed decorator detection for 3.11.0a5
      fix: detect decorators in 3.11.a6 correctly and ignore qualname checks for 3.11
      feat: added support for more bytecodes and worked around some issues
      moved implementation for 3.11 to NewNodeFinder
      added missing sample_results file for 3.11
      skip ast nodes in the which are related to annotations
      skip samples with incorrect syntax
      fixed some issues based on the current 3.11 branch
      wip: added _collections.py which leads to this bug python/cpython#95150 [skip ci]
      fix: fixed incompatibilities with master
      refactor: removed workaround for a fixed bug in python 3.11
      fix: removed limitation in tests where the reason was fixed in cpython
      fix: uncommented test_module_files
      refactor: moved some code and renamed both NodeFinders
      perf: removed call of co_positions
      fix: removed code without meaning
      refactor: removed some ifs which where never true and improved documentation
      refactor: applied changes requested in review and simplified code
      refactor: added instruction() and opname() methods
      fix: STORE_ATTR with multiline
      fix: DELETE_ATTR with multiline
      fix: multiline method calls
      test: limited new tests to python 3.11 and added some more linebreaks
      refactor: do attribute lookups always without lineno col_offset attribute comparison
      test: call more things
      fix: do not return Assert nodes, because they are caused by pytest assert rewriting
      fix: this allowed the profiling with cProfile but let the python 2.7 tests fail if you run them twice
      feat: cache get_instructions result
      fix: always try to find a node with all positions (required for some edge cases)
      fix: workaround for python/cpython#95921
      fix: EXTENDED_ARG is used for classes with lots of decorators
      fix: fixed some pattern matching related issues
      test: test with and listcomp
      fix: fix issues with master
      fix: handle assert-nodes as KnownIssue and limit the cases which can be fixed
      fix: handle exception cleanup for unmapped bytecodes
      fix: fix chain comparisons and report a KnownIssue if not possible
      fix: ignore pattern matching for now
      feat: implemented deadcode detection
      feat: implemented verifier
      test: improved tests
      refactor: extracted PositionNodeFinder into _position_node_finder.py
      refactor: used 3.11 syntax for _position_node_finder.py
      refactor: use lru_cache
      refactor: restructured special cases
      docu: explained __classcell__ issue
      refactor: splitted __init__
      refactor: created is_except_cleanup staticmethod
      refactor: simplified inst_match
      refactor: created parents helper method
      refactor: improved and tested mangled_name
      refactor: unified verification for % string formatting
      refactor: unified verification for function call and class definition
      doc: added missing todo
      refactor: use Tester to test with statement
      refactor: moved and renamed start() and end()
      fix: not can be found in some cases
      fix: `is/is not None`  can be found in some cases
      fix: removed duplicate code
      fix: fix `test_with` syntax for older python versions
      fix: specify python version
      fix: skip assert only for cpython 3.11
      refactor: removed unnecessary try
      refactor: some cleanup in _position_node_finder.py
      fix: extended tests and also mangle ExceptHandler.name
      test: deleted redundant test
      fix: check correct deadcode annotation and fixed implementation
      fix: with statement
      fix: use is to compare with True/False
      fix: renamed _case to case_
      fix: removed unused variables
      refactor: moved contains_break outside
      fix(deadcode): if condition is not dead based on the condition itself
      fix(verify): missing node for string concatenation
      fix: `while True:` can be broken by a `break` inside a `else:` block of a inner loop
      fix: __debug__ does not generate a LOAD instruction
      fix: some more `not ...` and `not x is x` do not generate instructions
      fix: classes where the mangeled names are in the bases do not count for name mangeling
      fix: not in `assert a if cnd else not b` can not be found, because it is part of the control flow
      fix: attribute gets not mangled if class name is `_`
      fix: tuples of literals are also literals
      fix: fixed bug in decorator detection
      fix: ExceptionHandler names can be mangled
      fix: also handle formatting cases like '%50s'%(a,)
      fix: `except TypeError as ValueError:` generated a STORE_GLOBAL
      fix: catch ValueError thrown by `ast.parse("\0")`
      fix: improved deadcode analysis
      build: moved test dependencies to setup.cfg
      test: extended test for multiline method calls
      fix: use previous non-CACHE instruction if f_lasti is a CACHE instruction
carterjsmarter pushed a commit to carterjsmarter/executing that referenced this issue Mar 14, 2024
carterjsmarter added a commit to carterjsmarter/executing that referenced this issue Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 only security fixes 3.12 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) release-blocker triaged The issue has been accepted as valid by a triager. type-bug An unexpected behavior, bug, or error
Projects
Development

No branches or pull requests

3 participants