Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: Cannot overwrite PYTHON_MODULE_EXTENSION when FindPython3.cmake is used #4854

Closed
3 tasks done
ilya-lavrenov opened this issue Sep 20, 2023 · 0 comments · Fixed by #4856
Closed
3 tasks done
Labels
triage New bug, unverified

Comments

@ilya-lavrenov
Copy link
Contributor

ilya-lavrenov commented Sep 20, 2023

Required prerequisites

What version (or hash if on master) of pybind11 are you using?

v2.11.1

Problem description

I'm trying to migrate our project from old FindPythonInterp and FindPythonLibs cmake modules to new FindPython3.
We are using cross-compilation in our project to compile ARM64 C++ and Python binaries.

In order to have a proper python extension for compiled modules, I used PYTHON_MODULE_EXTENSION. But with migration to new cmake python modules, it triggers different code path inside pybind11. In fact, PYTHON_MODULE_EXTENSION ignores my value set via cmake cmd.

This because of first run, pybind11 checks:

if(NOT ${_Python}_EXECUTABLE STREQUAL PYBIND11_PYTHON_EXECUTABLE_LAST)
# Detect changes to the Python version/binary in subsequent CMake runs, and refresh config if needed
unset(PYTHON_IS_DEBUG CACHE)
unset(PYTHON_MODULE_EXTENSION CACHE)
set(PYBIND11_PYTHON_EXECUTABLE_LAST
"${${_Python}_EXECUTABLE}"
CACHE INTERNAL "Python executable during the last CMake run")
endif()

While PYBIND11_PYTHON_EXECUTABLE_LAST is empty and unsets user-specified value for PYTHON_MODULE_EXTENSION. And later it sets PYTHON_MODULE_EXTENSION which results in build platform prefix:
if(NOT DEFINED PYTHON_MODULE_EXTENSION OR NOT DEFINED PYTHON_MODULE_DEBUG_POSTFIX)
execute_process(
COMMAND
"${${_Python}_EXECUTABLE}" "-c"
"import sys, importlib; s = importlib.import_module('distutils.sysconfig' if sys.version_info < (3, 10) else 'sysconfig'); print(s.get_config_var('EXT_SUFFIX') or s.get_config_var('SO'))"
OUTPUT_VARIABLE _PYTHON_MODULE_EXT_SUFFIX
ERROR_VARIABLE _PYTHON_MODULE_EXT_SUFFIX_ERR
OUTPUT_STRIP_TRAILING_WHITESPACE)
if(_PYTHON_MODULE_EXT_SUFFIX STREQUAL "")
message(
FATAL_ERROR "pybind11 could not query the module file extension, likely the 'distutils'"
"package is not installed. Full error message:\n${_PYTHON_MODULE_EXT_SUFFIX_ERR}"
)
endif()
# This needs to be available for the pybind11_extension function
if(NOT DEFINED PYTHON_MODULE_DEBUG_POSTFIX)
get_filename_component(_PYTHON_MODULE_DEBUG_POSTFIX "${_PYTHON_MODULE_EXT_SUFFIX}" NAME_WE)
set(PYTHON_MODULE_DEBUG_POSTFIX
"${_PYTHON_MODULE_DEBUG_POSTFIX}"
CACHE INTERNAL "")
endif()
if(NOT DEFINED PYTHON_MODULE_EXTENSION)
get_filename_component(_PYTHON_MODULE_EXTENSION "${_PYTHON_MODULE_EXT_SUFFIX}" EXT)
set(PYTHON_MODULE_EXTENSION
"${_PYTHON_MODULE_EXTENSION}"
CACHE INTERNAL "")
endif()
endif()

pybind11 has a cmake option PYBIND11_PYTHONLIBS_OVERWRITE, maybe we need to introduce something for FindPython3.cmake case? Or completely remove such under for the first cmake iteration.

Reproducible example code

See above

Is this a regression? Put the last known working version here if it is.

Not a regression

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage New bug, unverified
Projects
None yet
1 participant