Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

types.CodeType regression in Python 3.11 #100316

Closed
armoha opened this issue Dec 17, 2022 · 4 comments
Closed

types.CodeType regression in Python 3.11 #100316

armoha opened this issue Dec 17, 2022 · 4 comments
Labels
type-bug An unexpected behavior, bug, or error

Comments

@armoha
Copy link

armoha commented Dec 17, 2022

Bug report

# minimal reproducible example
import types
members = [  # 16 members
    "co_argcount", "co_posonlyargcount", "co_kwonlyargcount",
    "co_nlocals", "co_stacksize", "co_flags",
    "co_code", "co_consts", "co_names",
    "co_varnames", "co_filename", "co_name",
    "co_firstlineno", "co_lnotab", "co_freevars",
    "co_cellvars",
]
codeobj = (lambda x: x).__code__
newobj = types.CodeType(*(getattr(codeobj, member) for member in members))
# TypeError: code() argument 13 must be str, not int

I've already asked this on Python user discord server but couldn't get proper answer.

13th argument is codeobj.co_lnotab but putting invalid input (None, "" or b"") does not change the error message, TypeError: code() argument 13 must be str, not int. So I'm guessing there's a mismatch on the order of parameters of types.CodeType and code().

Context

I'm maintaining DSL, which can import Python and vice versa. I need to reconstruct lnotab for correct line numbers for e.g. stacktrace.
I tried using types.CodeType.replace but it does not support co_lnotab.
Original code: https://github.com/armoha/eudplib/blob/master/eudplib/epscript/epsimp.py#L46-L92

def modifyCodeLineno(codeobj, codeMap):
    co_lnotab = codeobj.co_lnotab
    co_firstlineno = codeobj.co_firstlineno

    # Reconstruct code data
    new_lnotab = []
    currentLine = co_firstlineno
    currentMappedLine = codeMap(currentLine)
    for i in range(0, len(co_lnotab), 2):
        bytecodeLen, lineAdvance = co_lnotab[i : i + 2]
        nextLine = currentLine + lineAdvance
        nextMappedLine = codeMap(nextLine)
        newLineAdvance = nextMappedLine - currentMappedLine
        while newLineAdvance >= 0xFF:
            new_lnotab.append(bytes([0, 0xFF]))
            newLineAdvance -= 0xFF
        new_lnotab.append(bytes([bytecodeLen, newLineAdvance]))
        currentLine = nextLine
        currentMappedLine = nextMappedLine

    # For code objects
    new_co_consts = []
    for c in codeobj.co_consts:
        if isinstance(c, types.CodeType):
            c = modifyCodeLineno(c, codeMap)
        new_co_consts.append(c)

    codeobj = types.CodeType(
        codeobj.co_argcount,
        codeobj.co_posonlyargcount,  # python 3.8 support (See PEP 570)
        codeobj.co_kwonlyargcount,
        codeobj.co_nlocals,
        codeobj.co_stacksize,
        codeobj.co_flags,
        codeobj.co_code,
        tuple(new_co_consts),
        codeobj.co_names,
        codeobj.co_varnames,
        codeobj.co_filename,
        codeobj.co_name,
        codeMap(co_firstlineno),  # codeobj.co_firstlineno,
        b"".join(new_lnotab),  # codeobj.co_lnotab,
        codeobj.co_freevars,
        codeobj.co_cellvars,
    )

    return codeobj

Your environment

  • CPython versions tested on: Python 3.11.1 (works fine on Python 3.7~10)
  • Operating system and architecture: Windows amd64
@armoha armoha added the type-bug An unexpected behavior, bug, or error label Dec 17, 2022
@thatbirdguythatuknownot
Copy link
Contributor

You seem to be missing co_qualname (after co_name) and co_exceptiontable (after co_lnotab). The last two in the list of attributes (co_cellvars and co_freevars) are optional and give you the illusion of 16 arguments.

@armoha
Copy link
Author

armoha commented Dec 17, 2022

Work like a charm! Thank you @thatbirdguythatuknownot

@armoha armoha closed this as completed Dec 17, 2022
@JelleZijlstra
Copy link
Member

But you should really be using CodeType.replace.

@armoha
Copy link
Author

armoha commented Dec 17, 2022

But you should really be using CodeType.replace.

Yeah I'd love to move but types.CodeType.replace(codeobj, co_lnotab=new_lnotab) raises error on Python 3.11 while it works fine on Python 3.9~3.10.
TypeError: 'co_lnotab' is an invalid keyword argument for replace()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

3 participants