Import error from bazel run :main_image

[statik]

Hi!

Thank you so much for putting this example together, and for contributing the python_interpreter_target support to rules_python. I am very excited about finding this example!

I'm hitting an error getting things to work. I have cloned this repo, and
bazel run :main works, but bazel run :main_image fails the import. I moved the import down to after the print statement in order to get a bit more info, and here is the error I get:

bazel run :main_image                                                                                                                                
INFO: Analyzed target //:main_image (7 packages loaded, 8248 targets configured).
INFO: Found 1 target...
Target //:main_image up-to-date:
  bazel-bin/main_image-layer.tar
INFO: Elapsed time: 233.890s, Critical Path: 13.38s
INFO: 17 processes: 17 darwin-sandbox.
INFO: Build completed successfully, 25 total actions
INFO: Build completed successfully, 25 total actions
de0f14108a99: Loading layer [==================================================>]  3.308MB/3.308MB
Loaded image ID: sha256:fb9a1886a7139a25ce0f52fbd0e6b4f7551c763188467a7c3808d1010aaeb3b2
Tagging fb9a1886a7139a25ce0f52fbd0e6b4f7551c763188467a7c3808d1010aaeb3b2 as bazel:main_image
Bazel python executable is /usr/bin/python3
Bazel python version is 3.8.3
Host python executable is /usr/local/bin/python3
Host python version is 3.8.3
Traceback (most recent call last):
  File "/app//main_image.binary.runfiles/py_test/main.py", line 22, in <module>
    import psycopg2
  File "/app/main_image.binary.runfiles/py_deps_pypi__psycopg2_binary_2_8_5/psycopg2/__init__.py", line 51, in <module>
    from psycopg2._psycopg import (                     # noqa
ModuleNotFoundError: No module named 'psycopg2._psycopg'

I'm using bazel 3.3.1, but have also tried with other bazel versions back to 2.2.0. I've also tried switching to a python 3.5.5 slim buster base image, but that didn't seem to make a difference.

When I run bazel run :main, here is the output I get

bazel run :main                                                      
DEBUG: Rule 'rules_python' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1592977862 -0700"
DEBUG: Repository rules_python instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule git_repository defined at:
  /private/var/tmp/_bazel_emurphy/b4aea5c375899709420e75e405383008/external/bazel_tools/tools/build_defs/repo/git.bzl:195:33: in <toplevel>
INFO: Analyzed target //:main (0 packages loaded, 1 target configured).
INFO: Found 1 target...
Target //:main up-to-date:
  bazel-bin/main
INFO: Elapsed time: 1.721s, Critical Path: 1.18s
INFO: 0 processes.
INFO: Build completed successfully, 4 total actions
INFO: Build completed successfully, 4 total actions
Bazel python executable is /private/var/tmp/_bazel_emurphy/b4aea5c375899709420e75e405383008/execroot/py_test/bazel-out/darwin-fastbuild/bin/main.runfiles/python_interpreter/python_bin
Bazel python version is 3.8.3
Host python executable is /usr/local/bin/python3
Host python version is 3.8.5
Successfully imported psycopg2-binary!

I have tested on MacOS 10.15.6 and Ubuntu 18.04.4 with the same results. Any tips or suggestions you can offer for getting this working are very appreciated.

[kku1993]

Thanks! I'm glad you found my example useful.

Regarding your issue, I think the problem is that rules_docker is picking up on the host's python version instead of the custom toolchain. One workaround that I've used in the past is to install external dependencies using pip when building the container image.

I've updated the repo to reflect that. Specifically, you may want to look at 0a595dc