-
Notifications
You must be signed in to change notification settings - Fork 111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ValueError: bad marshal data (unknown type code) #191
Comments
With Py3.11.3 on windows it "works-for-me" :-)
The last lines of the traceback shows:
which is calling python's standard You can likely find the problem-file with the (undocumented) (fwiw, the double-headed arrows indicate circular imports...) |
Thanks for the quick reply! Here are the last few lines of the debug output before the traceback I posted before:
Indeed, >>> import marshal
>>> f = open('/usr/lib64/python3.11/pydoc_data/topics.pyc', 'rb')
>>> marshal.load(f)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: bad marshal data (unknown type code) It's installed as a system library which I might try reinstalling, although it looks like a core Python library (
Yes, these are within some functions that convert from the objects in one class to the objects in another. |
I'm trying to reverse engineer the >>> import marshal
>>> f = open('/usr/lib64/python3.11/pydoc_data/topics.pyc', 'rb')
>>> f.read(12)
>>> marshal.load(f)
>>> f.close() compared to >>> import marshal
>>> f = open('/usr/lib64/python3.11/pydoc_data/topics.pyc', 'rb')
>>> f.read(8)
>>> marshal.load(f)
Traceback (most recent call last):
File "<string>", line 1, in <module>
ValueError: bad marshal data (unknown type code) |
It should be the Python "magic number":
This path
|
This file is in the OS packages and is definitely a bit odd. There is also a
but it only contains the
It's provided in the I'll raise this as an issue in the Fedora package. At a glance it looks like the only file provided as a |
Yes, that sounds like a packaging issue, the topics.py file should definitely be present:
topics.py starts with:
so maybe a sphinx step was omitted? |
I've emailed the maintainers of the Fedora package. 📦 |
Hello. I am one of Fedora's Python maintainers. topics and encodings are shipped as .pyc-only on purpose. The files are generated and we decided to only ship bytecode to save disk space. See https://src.fedoraproject.org/rpms/python3.11/c/740668aab7abe02f47d7a69e800c61b8b5e52f51 We have never encountered problems with that. It is a supported way to ship Python modules. What is the code here trying to do? |
@hroncok The problem seems to be that the |
Yes, it's not in the pycache folder, but why would you think the magic number is incorrect? How do I quickly check the number from Python or shell to see if that's the case? |
I see the comments above. Will debug the headers. However note that I just got back from EuroPython and I am taking some time off computers. |
@hroncok No worries, I'm on summer vacation myself :-) |
>>> import pathlib, struct, marshal
>>> pyc1 = pathlib.Path('/usr/lib64/python3.11/encodings/cp1250.pyc')
>>> pyc2 = pathlib.Path('/usr/lib64/python3.11/encodings/__pycache__/cp1125.cpython-311.pyc')
>>> bytes1 = pyc1.read_bytes()
>>> bytes2 = pyc2.read_bytes()
>>> bytes1[:4]
b'\xa7\r\r\n'
>>> bytes2[:4]
b'\xa7\r\r\n'
>>> struct.unpack("<H2B", bytes1[:4])
(3495, 13, 10) 3495 is Python 3.11b4+, see importlib/_bootstrap_external.py PEP 552 says:
That's 16 bytes for Python 3.7+: >>> marshal.loads(bytes1[16:])
<code object <module> at 0x7fb1a73f9a70, file "/usr/lib64/python3.11/encodings/cp1250.py", line 1>
>>> marshal.loads(bytes2[16:])
<code object <module> at 0x558a998382f0, file "/usr/lib64/python3.11/encodings/cp1125.py", line 1> This is consistent with files in What seems to be the problem here? |
I believe this comment is outdated: Lines 66 to 69 in 3c1c40b
The number of bytes at point 2 depends on the Python version.
The number is hardcocded here: Line 74 in 3c1c40b
If you care only for Pythons that are not yet end of life, changing this number to 12 should do the trick. |
Looks like your analysis is correct (I'm still on vacation, so I haven't investigated why there isn't a problem on windows...). The native modulefinder calls 23e0a17 is a (WIP) version that accounts for the different sizes. How to test it isn't immediately obvious to my vacation brain, but I'm sure I'll figure it out soon ;-) |
I'm running into this same problem using CentOS Stream 9 with I don't have this issue for personal projects that run in GitHub CI though (on Seems to corroborate that this is some RHEL thing. (edit: I'm not caught up with the details on the thread, seems like there's a "why", just noting a +1 to problem bisection) |
Yes, CentOS has the same pyc files. |
v1.12.17 should have fixed .pyc header parsing code. |
I just decided to try pydeps on a project of mine but it fails with the error (full error message at the end):
This is pydeps 1.12.12 installed via pip on Fedora 38 with Python 3.11.4 installed via system packages.
I tried a few things suggested in #181 to no avail. All the files are in the
tomso
folder and there is a__init__.py
file.Full error message
The text was updated successfully, but these errors were encountered: