Issue Running Python Application with Bazel - ImportError with NumPy C-Extensions

[ModiCodeCraftsman]

🐞 bug report

Affected Rule

The issue is caused by the rule: py_binary

Is this a regression?

No

Description

When running a py_binary target with dependencies that include NumPy and other libraries requiring C-extensions, the runtime fails to locate the required C-extension libraries. This results in an ImportError for NumPy's C-extensions, even though the same environment works correctly when run outside of Bazel in a local virtual environment.

🔬 Minimal Reproduction

Minimal Reproduction Repository

1. Clone the repository from the provided link.
2. Run bazel run //inference:run_app.

🔥 Exception or Error

INFO: Analyzed target //inference:run_app (1 packages loaded, 4 targets configured).
INFO: Found 1 target...
Target //inference:run_app up-to-date:
  bazel-bin/inference/run_app
INFO: Elapsed time: 1.159s, Critical Path: 0.69s
INFO: 4 processes: 4 internal.
INFO: Build completed successfully, 4 total actions
INFO: Running command line: bazel-bin/inference/run_app
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_modit/96245a706bfbcbe234af177b6d166fc2/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/inference/run_app.runfiles/infaas_deps_numpy/site-packages/numpy/core/__init__.py", line 24, in 
    from . import multiarray
  File "/private/var/tmp/_bazel_modit/96245a706bfbcbe234af177b6d166fc2/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/inference/run_app.runfiles/infaas_deps_numpy/site-packages/numpy/core/multiarray.py", line 10, in 
    from . import overrides
  File "/private/var/tmp/_bazel_modit/96245a706bfbcbe234af177b6d166fc2/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/inference/run_app.runfiles/infaas_deps_numpy/site-packages/numpy/core/overrides.py", line 8, in 
    from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

🌍 Your Environment

Operating System:

  
Ubuntu 22.04
  

  
  Bazelisk version: 1.20.0
  Build label: 7.1.0
  Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
  Build time: Mon Mar 11 17:55:51 2024 (1710179751)
  Build timestamp: 1710179751
  Build timestamp as int: 1710179751
  

  
SHA="f4770504f74b9ef87984224a06df35b931d35a2e695ea898102bd9df30ce4fe6"
  VERSION="0.32.0"

  http_archive(
      name = "rules_python",
      sha256 = SHA,
      strip_prefix = "rules_python-{}".format(VERSION),
      url = "https://github.com/bazelbuild/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(VERSION,VERSION),
  )
  

[jvolkman]

I think you're running into #1675. I see that your example is setup to use Python 3.11, but when I run it on my machine, numpy says I'm running 3.12 which is the host version.

Either disabling bzlmod (--noenable_bzlmod) or moving to the bzlmod Python rules resolves the issue. For the latter, clear out the WORKSPACE file and add the following to MODULE.bazel.

bazel_dep(name = "rules_python", version = "0.32.0")

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
    python_version = "3.11",
)

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
    hub_name = "infaas_deps",
    python_version = "3.11",
    requirements_lock = "//inference:requirements.txt",
)
use_repo(pip, "infaas_deps")

You should also enable hashes when compiling requirements.txt.

[github-actions]

This issue was automatically closed because it went 30 days without a reply since it was labeled "Can Close?"