cantera fails to build with Python 3.11: ValueError: invalid mode: 'rU' #1325

mefuller · 2022-06-07T18:55:31Z

Reported in full at https://bugzilla.redhat.com/show_bug.cgi?id=2094258

Problem description

cantera fails to build with Python 3.11.0b3.

Steps to reproduce

ValueError: invalid mode: 'rU':
File "/builddir/build/BUILD/cantera-2.6.0/SConstruct", line 2020:
SConscript("build/python/SConscript")
File "/usr/lib/python3.11/site-packages/SCons/Script/SConscript.py", line 660:
return method(*args, **kw)
File "/usr/lib/python3.11/site-packages/SCons/Script/SConscript.py", line 597:
return _SConscript(self.fs, *files, **subst_kw)
File "/usr/lib/python3.11/site-packages/SCons/Script/SConscript.py", line 285:
exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
File "/builddir/build/BUILD/cantera-2.6.0/build/python/SConscript", line 98:
setup_cfg = localenv.SubstFile("setup.cfg", "setup.cfg.in")
File "/usr/lib/python3.11/site-packages/SCons/Util.py", line 742:
return self.method(*nargs, **kwargs)
File "/builddir/build/BUILD/cantera-2.6.0/site_scons/site_tools/subst.py", line 154:
return env.SubstGeneric(target,
File "/usr/lib/python3.11/site-packages/SCons/Environment.py", line 238:
return super().call(target, source, *args, **kw)
File "/usr/lib/python3.11/site-packages/SCons/Util.py", line 742:
return self.method(*nargs, **kwargs)
File "/usr/lib/python3.11/site-packages/SCons/Builder.py", line 662:
return self._execute(env, target, source, OverrideWarner(kw), ekw)
File "/usr/lib/python3.11/site-packages/SCons/Builder.py", line 568:
tlist, slist = self._create_nodes(env, target, source)
File "/usr/lib/python3.11/site-packages/SCons/Builder.py", line 530:
target, source = self.emitter(target=tlist, source=slist, env=env)
File "/builddir/build/BUILD/cantera-2.6.0/site_scons/site_tools/subst.py", line 124:
keys = _subst_keys(path, pattern)
File "/builddir/build/BUILD/cantera-2.6.0/site_scons/site_tools/subst.py", line 49:
f = open(source, "rU")
error: Bad exit status from /var/tmp/rpm-tmp.S5BgPF (%build)

open(), io.open(), codecs.open() and fileinput.FileInput no longer accept 'U' (“universal newline”) in the file mode. This flag was deprecated since Python 3.3. In Python 3, the “universal newline” is used by default when a file is open in text mode. The newline parameter of open() controls how universal newlines works. (Contributed by Victor Stinner in bpo-37330.)

https://bugs.python.org/issue?@action=redirect&bpo=37330
https://docs.python.org/3.11/whatsnew/3.11.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.11/fedora-rawhide-x86_64/04495077-cantera/

For all our attempts to build cantera with Python 3.11, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/package/cantera/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.11:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/

Let us know here if you have any questions.

Python 3.11 is planned to be included in Fedora 37. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.11.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

System information

Cantera version: 2.6.0 release
OS: Fedora Linux 37/rawhide
Python/MATLAB/other software versions: Python 3.11b

mefuller · 2022-06-07T19:56:11Z

I fixed the above error in (draft) PR #1326, but then we get another error:

/usr/bin/python3 -c "import Cython.Build; Cython.Build.cythonize('build/python/cantera/_cantera.pyx')"
Compiling build/python/cantera/_cantera.pyx because it changed.
[1/1] Cythonizing build/python/cantera/_cantera.pyx
g++ -o build/temp-py/_cantera.os -c -std=c++11 -pthread -O3 -Wno-inline -g -fPIC -DNDEBUG -Iinclude -I/usr/include/python3.11 -I/usr/lib64/python3.11/site-packages/numpy/core/include -I/usr/include/eigen3 build/python/cantera/_cantera.cpp
In file included from /usr/lib64/python3.11/site-packages/numpy/core/include/numpy/ndarraytypes.h:1960,
                 from /usr/lib64/python3.11/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /usr/lib64/python3.11/site-packages/numpy/core/include/numpy/arrayobject.h:5,
                 from build/python/cantera/_cantera.cpp:749:
/usr/lib64/python3.11/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~
In file included from /usr/include/python3.11/Python.h:38,
                 from build/python/cantera/_cantera.cpp:20:
/usr/include/python3.11/pyport.h: In instantiation of 'type {anonymous}::_Py_CAST_impl(const expr_type&) [with type = _object*; expr_type = int]':
build/python/cantera/_cantera.cpp:193144:9:   required from here
/usr/include/python3.11/pyport.h:47:24: error: invalid 'static_cast' from type 'int' to type '_object*'
   47 |                 return static_cast<type>(const_cast<expr_type &>(expr));
      |                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scons: *** [build/temp-py/_cantera.os] Error 1
scons: building terminated because of errors.

ischoegl · 2022-06-07T20:05:54Z

For a recent discussion on Python 3.11, see #1287 ... I recall that there were some issues, and things were tabled. If you fix this, 3.11 should likely be added to CI (as long as the GH version is sufficiently recent). Tagging @bryanwweber on this ...

Here are some build logs with failures ... as things ran farther on Ubuntu/macOS/Windows, I am wondering whether there are any additional fedora-specific issues at play.

bryanwweber · 2022-06-07T20:30:47Z

The error in this message appears to be to do with Cython, which will have to be updated for Python 3.11. They may or may not have backported any required fixes to a 0.* release, or maybe it's only on main (their 3.0.0a*), or even maybe it's not been fixed yet 🤷‍♂️

mefuller · 2022-06-07T20:35:16Z

@ischoegl thanks - I wasn't aware of the previous discussion, but I feel pretty well caught up now.
@bryanwweber Fedora has a python 3.11 rebuild repo for testing: https://copr.fedorainfracloud.org/coprs/g/python/python3.11/, so the preceding is with a cython rebuilt against python 3.11

I'm building with the 2.6.0 release source code with only the modification in #1326 applied

We're anticipating running with 3.11 on Fedora 37, to be released in <=6 months, so my personal hope is to be able to build cantera 2.6 with python 3.11 by about 4 months from now.

bryanwweber · 2022-06-07T22:22:35Z

so the preceding is with a cython rebuilt against python 3.11

It's not just that Cython needs to be rebuilt, it sometimes needs to be updated in terms of the C++ code it generates to match changes in the Python C-API. If that hasn't been done yet, there's nothing we can do until they do. I'm not sure that's the problem, but since the error is due to an include file from Python itself, that's my interpretation of the error.

hrnciar · 2022-06-21T07:03:58Z

The error in this message appears to be to do with Cython, which will have to be updated for Python 3.11. They may or may not have backported any required fixes to a 0.* release, or maybe it's only on main (their 3.0.0a*), or even maybe it's not been fixed yet man_shrugging

Cython was updated for Python 3.11 on the main branch (3.0.0a*) and necessary commits were backported to 0.x branch. In Fedora we have Cython 0.29.30 which should be compatible with Python 3.11. If Canteras is broken by Cython it might be possible that there is some bug that needs to be fixed on Cython's side. Could you please open an issue there?

hroncok · 2022-06-21T10:02:01Z

Possibly this is python/cpython#93442

mefuller · 2022-06-21T20:44:34Z

Possibly this is python/cpython#93442

It is exactly that.
The latter issue is fixed by python/cpython#93442

@bryanwweber @ischoegl The original issue regarding deprecated 'rU' is the only issue in the Cantera source code currently preventing builds under Python 3.11
PR #1326 would close this issue, despite not being a comprehensive update for Python 3.11

ischoegl added Python compiling labels Jun 7, 2022

mefuller mentioned this issue Jun 21, 2022

drop deprecated 'U' open option and fix formatting (Python 3.11 build errors) #1326

Merged

5 tasks

bryanwweber closed this as completed in #1326 Jun 23, 2022

dreamcat4 mentioned this issue Aug 12, 2023

[python-3.11] open() - invalid mode 'rU' Tuhinshubhra/ExtAnalysis#30

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Sponsors

cantera fails to build with Python 3.11: ValueError: invalid mode: 'rU' #1325

cantera fails to build with Python 3.11: ValueError: invalid mode: 'rU' #1325

mefuller commented Jun 7, 2022

mefuller commented Jun 7, 2022

ischoegl commented Jun 7, 2022 •

edited

Loading

bryanwweber commented Jun 7, 2022

mefuller commented Jun 7, 2022

bryanwweber commented Jun 7, 2022

hrnciar commented Jun 21, 2022

hroncok commented Jun 21, 2022

mefuller commented Jun 21, 2022

cantera fails to build with Python 3.11: ValueError: invalid mode: 'rU' #1325

cantera fails to build with Python 3.11: ValueError: invalid mode: 'rU' #1325

Comments

mefuller commented Jun 7, 2022

mefuller commented Jun 7, 2022

ischoegl commented Jun 7, 2022 • edited Loading

bryanwweber commented Jun 7, 2022

mefuller commented Jun 7, 2022

bryanwweber commented Jun 7, 2022

hrnciar commented Jun 21, 2022

hroncok commented Jun 21, 2022

mefuller commented Jun 21, 2022

ischoegl commented Jun 7, 2022 •

edited

Loading