Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runtime error python3xy.dll not found: Missing srcs in python_repository's libpython rule for os:windows #1823

Open
axbycc-mark opened this issue Mar 24, 2024 · 4 comments
Labels
type: bug type: toolchain Related to the toolchains provided by rules_python

Comments

@axbycc-mark
Copy link

axbycc-mark commented Mar 24, 2024

🐞 bug report

The srcs attribute for this cc_library for os:windows is ["python3.dll", "libs/python{python_version_nodot}.lib"]. It is missing the version-specific binary, eg python311.dll.

cc_library(
name = "libpython",
hdrs = [":includes"],
srcs = select({{
"@platforms//os:windows": ["python3.dll", "libs/python{python_version_nodot}.lib"],
"@platforms//os:macos": ["lib/libpython{python_version}.dylib"],
"@platforms//os:linux": ["lib/libpython{python_version}.so", "lib/libpython{python_version}.so.1.0"],
}}),
)

This will result in a runtime error (python311.dll not found) in the following scenario.

  1. User creates a cc_binary which has #include <Python.h> and which depends on the libpython above.
  2. User never defines #Py_LIMITED_ABI
  3. User attempts to run the cc_binary

The Python.h includes the pyconfig.h, which causes msvc to attempt linking against python3xy.dll via the#pragma comment(lib, ...)mechanism. The relevant code from the cpython project is here.
https://github.com/python/cpython/blob/d610d821fd210dce63a1132c274ffdf8acc510bc/PC/pyconfig.h.in#L315-L331

I have resolved this locally by globbing *.dll into the libpython cc_library.

A similar issue occurs if the user defines #Py_DEBUG, which causes msvc to attempt linking against the debug binary python3xy_d.dll. I did not see a way to request debug binaries using the python_repository rule but if that were enabled, I think globbing *.dll would fix that case too.

Affected Rule

python_repository

Is this a regression?

Unsure.

Description

See above.

🔬 Minimal Reproduction

See above.

🔥 Exception or Error

Runtime error python3xy.dll not found

🌍 Your Environment

Operating System:

  
Windows 10
  

Output of bazel version:

  
7.0.2
  

Rules_python version:

  
0.31.0
  

Anything else relevant?

@axbycc-mark axbycc-mark changed the title Missing srcs in python_repository's libpython rule for os:windows Runtime error python3xy.dll not found: Missing srcs in python_repository's libpython rule for os:windows Mar 24, 2024
@aignas aignas added type: bug type: toolchain Related to the toolchains provided by rules_python labels Mar 24, 2024
@axbycc-mark
Copy link
Author

Possibly related test failure

# Mac and Windows fail with linking errors, but its not clear why; someone
# with more C + Mac/Windows experience will have to figure it out.
# - rickeylev@

@nicholasjng
Copy link
Contributor

For the same reason as in this example, statically linking a Python extension on Windows fails when Py_LIMITED_API is defined because the unversioned libs/python3.lib is not in that list. That is fixed in #1820, depending on the feedback on that PR, I suppose I could easily integrate this case there as well.

@axbycc-mark
Copy link
Author

axbycc-mark commented Mar 26, 2024

Similarly, any cc_library depending on @rules_python//python/cc:current_py_cc_headers and including <Python.h> will also break when compiled with -c dbg, since this also causes Python.h to attempt to link to the python3[xy]_d.lib, which is not available.

Copy link

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

@github-actions github-actions bot added the Can Close? Will close in 30 days if there is no new activity label Sep 22, 2024
@aignas aignas removed the Can Close? Will close in 30 days if there is no new activity label Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug type: toolchain Related to the toolchains provided by rules_python
Projects
None yet
Development

No branches or pull requests

3 participants