Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Build failure with Python 3.11 #58

Closed
opoplawski opened this issue Jul 6, 2022 · 6 comments
Closed

[BUG] Build failure with Python 3.11 #58

opoplawski opened this issue Jul 6, 2022 · 6 comments
Labels
Type: Bug Something isn't working

Comments

@opoplawski
Copy link

Welcome! You should write in your Bug Report:

OS version (https://www.google.com/search?channel=fs&q=check+os+version&ie=utf-8&oe=utf-8):
Fedora Rawhide

Python3 version (python3 -V -V):
Python 3.11.0b3 (main, Jun 24 2022, 00:00:00) [GCC 12.1.1 20220507 (Red Hat 12.1.1-1)]

Actual result (with the python stack trace if present):

+ /usr/bin/python3 setup.py build '--executable=/usr/bin/python3 -s'
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/frozendict
copying frozendict/__init__.py -> build/lib.linux-x86_64-cpython-311/frozendict
copying frozendict/core.py -> build/lib.linux-x86_64-cpython-311/frozendict
copying frozendict/VERSION -> build/lib.linux-x86_64-cpython-311/frozendict
copying frozendict/py.typed -> build/lib.linux-x86_64-cpython-311/frozendict
running build_ext
building 'frozendict._frozendict' extension
creating build/temp.linux-x86_64-cpython-311
creating build/temp.linux-x86_64-cpython-311/frozendict
creating build/temp.linux-x86_64-cpython-311/frozendict/src
creating build/temp.linux-x86_64-cpython-311/frozendict/src/3_11
gcc -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/Include -I/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects -I/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/stringlib -I/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/clinic -I/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src -I/usr/include/python3.11 -c frozendict/src/3_11/frozendictobject.c -o build/temp.linux-x86_64-cpython-311/frozendict/src/3_11/frozendictobject.o -DPY_SSIZE_T_CLEAN
cc1: fatal error: frozendict/src/3_11/frozendictobject.c: No such file or directory
compilation terminated.
@opoplawski opoplawski added the Type: Bug Something isn't working label Jul 6, 2022
@opoplawski
Copy link
Author

If I copy 3_10 to 3_11:

In file included from frozendict/src/3_11/frozendictobject.c:6:
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘_PyDict_CheckConsistency’:
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:491:36: error: invalid use of undefined type ‘struct _dictvalues’
  491 |                 CHECK(mp->ma_values[i] != NULL);
      |                                    ^
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:434:16: note: in definition of macro ‘CHECK’
  434 |     do { if (!(expr)) { _PyObject_ASSERT_FAILED_MSG(op, Py_STRINGIFY(expr)); } } while (0)
      |                ^~~~
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:491:23: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
  491 |                 CHECK(mp->ma_values[i] != NULL);
      |                       ^~
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:434:16: note: in definition of macro ‘CHECK’
  434 |     do { if (!(expr)) { _PyObject_ASSERT_FAILED_MSG(op, Py_STRINGIFY(expr)); } } while (0)
      |                ^~~~
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘_d_PyDict_Next’:
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:799:30: error: invalid use of undefined type ‘struct _dictvalues’
  799 |         value = mp->ma_values[i];
      |                              ^
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:799:17: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
  799 |         value = mp->ma_values[i];
      |                 ^~
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘dict_dealloc’:
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:830:25: warning: initialization of ‘PyObject **’ {aka ‘struct _object **’} from incompatible pointer type ‘PyDictValues *’ {aka ‘struct _dictvalues *’} [-Wincompatible-pointer-types]
  830 |     PyObject **values = mp->ma_values;
      |                         ^~
In file included from /usr/include/python3.11/Python.h:45,
                 from frozendict/src/3_11/frozendictobject.c:1:
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘dict_traverse’:
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:39: error: invalid use of undefined type ‘struct _dictvalues’
 1090 |                 Py_VISIT(mp->ma_values[i]);
      |                                       ^
/usr/include/python3.11/objimpl.h:199:13: note: in definition of macro ‘Py_VISIT’
  199 |         if (op) {                                                       \
      |             ^~
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:26: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
 1090 |                 Py_VISIT(mp->ma_values[i]);
      |                          ^~
/usr/include/python3.11/objimpl.h:199:13: note: in definition of macro ‘Py_VISIT’
  199 |         if (op) {                                                       \
      |             ^~
In file included from /usr/include/python3.11/Python.h:38:
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:39: error: invalid use of undefined type ‘struct _dictvalues’
 1090 |                 Py_VISIT(mp->ma_values[i]);
      |                                       ^
/usr/include/python3.11/pyport.h:69:40: note: in definition of macro ‘_Py_CAST’
   69 | #  define _Py_CAST(type, expr) ((type)(expr))
      |                                        ^~~~
/usr/include/python3.11/objimpl.h:200:30: note: in expansion of macro ‘_PyObject_CAST’
  200 |             int vret = visit(_PyObject_CAST(op), arg);                  \
      |                              ^~~~~~~~~~~~~~
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:17: note: in expansion of macro ‘Py_VISIT’
 1090 |                 Py_VISIT(mp->ma_values[i]);
      |                 ^~~~~~~~
/usr/include/python3.11/pyport.h:69:39: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
   69 | #  define _Py_CAST(type, expr) ((type)(expr))
      |                                       ^
/usr/include/python3.11/object.h:107:28: note: in expansion of macro ‘_Py_CAST’
  107 | #define _PyObject_CAST(op) _Py_CAST(PyObject*, (op))
      |                            ^~~~~~~~
/usr/include/python3.11/objimpl.h:200:30: note: in expansion of macro ‘_PyObject_CAST’
  200 |             int vret = visit(_PyObject_CAST(op), arg);                  \
      |                              ^~~~~~~~~~~~~~
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:17: note: in expansion of macro ‘Py_VISIT’
 1090 |                 Py_VISIT(mp->ma_values[i]);
      |                 ^~~~~~~~
frozendict/src/3_11/frozendictobject.c: In function ‘frozendict_merge’:
frozendict/src/3_11/frozendictobject.c:447:41: error: invalid use of undefined type ‘struct _dictvalues’
  447 |                 value = other->ma_values[i];
      |                                         ^
frozendict/src/3_11/frozendictobject.c:447:25: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
  447 |                 value = other->ma_values[i];
      |                         ^~~~~
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c: At top level:
/home/orion/fedora/python-frozendict/frozendict-2.3.2/frozendict/src/3_11/cpython_src/Objects/dictobject.c:431:1: warning: ‘_PyDict_CheckConsistency’ defined but not used [-Wunused-function]
  431 | _PyDict_CheckConsistency(PyObject *op, int check_content)
      | ^~~~~~~~~~~~~~~~~~~~~~~~

@Marco-Sulla
Copy link
Owner

Hello,

beta, alpha and non-release versions of Python and PyPy are not supported. You can just use the pyre python package.

@opoplawski
Copy link
Author

FWIW - I'm am packaging frozendict for Fedora and it (rawhide specifically) is moving to 3.11 to be ready for when it is released. So the pyre python package is not relevant here.

@Marco-Sulla
Copy link
Owner

Well, I'm quite sure this is exciting for much people.Anyway, two considerations.

  1. Rawhide, for what I know, is the current dev branch of Fedora. I think it's normal for a dev branch to use beta packages, in particular for Fedora
  2. Python 3.11 seems to have changed the code of dict a lot internally. Inada and the other core devs did a great job. Since the C Extension of frozendict uses a bunch of heavily modified dict source code, the adaptation is not trivial this time. Furthermore the dict code of Python 3.11, IMHO, it's quite better, and that means that probably I'll rebase the code of all the other versions to 3.11. This is to make it easier modification of the C code for all the supported version (currently, the base code is the 3.10 one).

Shortly, the effort of making a frozendict C Extension for Python 3.11 seems to be harder than the previous versions, and I don't want to risk to make the extension and see the CPython 3.11 dict code change again under my nose.

Is it too much for you waiting 5-6 months for the official stable release?

@opoplawski
Copy link
Author

It appears that would be fine - Fedora is actually still on frozendict 1.2 which seems to build fine with Python 3.11.

@Marco-Sulla
Copy link
Owner

So I don't see why you shouldn't use the py version of frozendict 2 in the meanwhile. Frozendict 1.2 is pure python too, requires patches to work because is unmantained, it's more slow, less immutable and with a naif hash algorithm.

The py version of frozendict 2 can be packaged using

python3 setup.py py bdist_wheel

Don't know how you can specify it in you rpm.

@lurch lurch mentioned this issue Sep 22, 2022
Closed
@sgn sgn mentioned this issue Sep 22, 2022
Closed
@Marco-Sulla Marco-Sulla mentioned this issue Oct 15, 2022
Closed
arkamar added a commit to arkamar/python-frozendict that referenced this issue Dec 3, 2022
@arkamar
Autodetect availability of c extension
3488269 
This improves default behavior of setup.py in sense that py version of
frozendict is used for python versions which do not have C extension
available yet. This would probably reduce amount of issues similar to
[1-4] in future.

[1] Marco-Sulla#68
[2] Marco-Sulla#67
[3] Marco-Sulla#65
[4] Marco-Sulla#58
arkamar added a commit to arkamar/python-frozendict that referenced this issue Dec 3, 2022
@arkamar
Autodetect availability of C extension
5990c09 
This improves default behavior of setup.py in sense that py version of
frozendict is used for python versions which do not have C extension
available yet. This would probably reduce amount of issues similar to
[1-4] in future.

[1] Marco-Sulla#68
[2] Marco-Sulla#67
[3] Marco-Sulla#65
[4] Marco-Sulla#58
@arkamar arkamar mentioned this issue Dec 3, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@opoplawski @Marco-Sulla