ImportError: No module named moves.urllib.parse

[raulAtNines]

Hi,

I am trying to publish a java image from our bazel 0.28.1 build. I can build the image locally without issues with bazel run mytarget. However, when I try to publish the image with container_push, I run into an issue with six:

Traceback (most recent call last):
  File "/mypath/bazel-out/host/bin/external/containerregistry/digester.runfiles/containerregistry/tools/image_digester_.py", line 28, in <module>
    from containerregistry.client.v2_2 import docker_image as v2_2_image
  File "/mypath/bazel-out/host/bin/external/containerregistry/digester.runfiles/containerregistry/client/__init__.py", line 19, in <module>
    from containerregistry.client import docker_name_
  File "/mypath/bazel-out/host/bin/external/containerregistry/digester.runfiles/containerregistry/client/docker_name_.py", line 23, in <module>
    import six.moves.urllib.parse
ImportError: No module named moves.urllib.parse
----------------
Note: The failure of target @containerregistry//:digester (with exit code 1) may have been caused by the fact that it is running under Python 2 instead of Python 3. Examine the error to determine if that appears to be the problem. Since this target is built in the host configuration, the only way to change its version is to set --host_force_python=PY3, which affects the entire build.

If this error started occurring in Bazel 0.27 and later, it may be because the Python toolchain now enforces that targets analyzed as PY2 and PY3 run under a Python 2 and Python 3 interpreter, respectively. See https://github.com/bazelbuild/bazel/issues/7899 for more information.
----------------

Earlier today I got the environment to work with PY2, which I think is required here. However, I tried with PY3 and the issue was similar:

Traceback (most recent call last):
  File "/mypath/bazel-out/host/bin/external/containerregistry/digester.runfiles/containerregistry/tools/image_digester_.py", line 28, in <module>
    from containerregistry.client.v2_2 import docker_image as v2_2_image
  File "/mypath/bazel-out/host/bin/external/containerregistry/digester.runfiles/containerregistry/client/__init__.py", line 19, in <module>
    from containerregistry.client import docker_name_
  File "/mypath/bazel-out/host/bin/external/containerregistry/digester.runfiles/containerregistry/client/docker_name_.py", line 23, in <module>
    import six.moves.urllib.parse
ImportError: No module named moves.urllib.parse

Except the PY3 message goes away.

Any hints on how to work around this? Is this something for containerregistry to fix?

Thanks,
Raul.

[nlopezgi]

You currently need to use --host_force_python=PY2 for these rules to work. Is this error occurring even if you use this flag?
See here for full discussion of status regarding status of our dependency on containerregsitry #842

[raulAtNines]

Thanks for the prompt reply, Nicolas.

Indeed, I am using:

build --incompatible_use_python_toolchains
build --host_force_python=PY2
build --incompatible_py3_is_default=false

and

run --incompatible_use_python_toolchains
run --host_force_python=PY2
run --incompatible_py3_is_default=false

I believe that is my current setup. I had to purposely set PY3, and the message about python 2 disappeared, but the error would still persist.

I can certainly double check and report back tomorrow; I do not have access to the build system at the moment.

[raulAtNines]

Earlier today I had the same error as #842 locally (python3 syntax), and I had to update our host environment to run PY2, with py_test runtimes set to PY3 for our code. That's what made me think that container_push should also work with the PY2 host, as you confirmed. However, I am getting the error. Building the image does work locally.

[nlopezgi]

we dont set the incompatible_py3_is_default=false flag in our builds, can you try without that flag?

[raulAtNines]

Tried all combinations, with no luck:

#run --incompatible_use_python_toolchains
run --host_force_python=PY2
#run --incompatible_py3_is_default=false

I am setting those params in .bazelrc, and they do seem to have some effect.
My command is bazel run //mypath:publish, where publish:

...
    container_push(
        name = "publish",
        image = image,
        format = "Docker",
        registry = REGISTRY,
        repository = "{repository}/{name}".format(repository = REPOSITORY, name = name),
        tag = "{BUILD_SCM_REVISION}",
    )

[raulAtNines]

Let me try destroying the cache.

[raulAtNines]

Same error. Maybe related to #55 ?
We are using other libraries that depend on six:

Collecting six>=1.9 (from protobuf==3.6.1->-r /userpath/.cache/bazel/29e2a8bc0e6b2a4bc5bd63a8ff5efa60/external/build_stack_rules_proto/python/requirements/protobuf.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
  Saved ./six-1.12.0-py2.py3-none-any.whl

  Collecting six>=1.5.2 (from grpcio==1.20.0->-r /userpath/.cache/bazel/29e2a8bc0e6b2a4bc5bd63a8ff5efa60/external/build_stack_rules_proto/python/requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
  Saved ./six-1.12.0-py2.py3-none-any.whl

  Collecting six>=1.5.2 (from grpcio==1.19.0->-r /userpath/requirements.txt (line 2))
    Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
    Saved ./six-1.12.0-py2.py3-none-any.whl

[raulAtNines]

@nlopezgi
When I look in our host/bin/external, I see host/bin/external/six/six.py,

This suggests suggest that it should be renamed to __init__.py? (@mattmoor)
https://github.com/bazelbuild/rules_docker/blob/master/repositories/repositories.bzl#L161-L182

Other than porting to PY2, which I already did when we started using docker_rules, I have no ideas at the moment on how to fix the six issue, so if anyone can give me a hint, I would appreciate it.

Otherwise plan B is to push docker images to the registry as a separate build step.

Thanks.

[raulAtNines]

/host/bin/external/containerregistry/digester.runfiles/six $ ls

0 -r-xr-xr-x   1 raul  staff    0 Jul 25 13:23 __init__.py
0 lrwxr-xr-x   1 raul  staff  149 Jul 25 13:23 six.py -> symlink

[raulAtNines]

I was finally able to make it work by manually editing:

bazel-out/host/bin/external/containerregistry/digester.runfiles/six/__init__.py

required by digester,
and

bazel-out/darwin-py2-fastbuild/bin/..myTarget../publish.runfiles/six/__init__.py

required by publish.

As noticed above, the symlink does not work for __init__.py in either folder:

0 -r-xr-xr-x   1 raul  staff    0 Jul 29 17:04 __init__.py
0 lrwxr-xr-x   1 raul  staff  149 Jul 29 17:04 six.py -> /.../bazel-out/host/bin/external/six/six.py

The manual fix is to copy (or link) sudo cp six.py __init__.py into __init__.py.

Now I need to investigate how those symlinks are generated. Is that part of the bazel sandboxing process?

[laurit17]

I am in an identical situation, depending on some other libraries that use six (though I am exclusively using python 2.7 at this point), and I suspect that copy from six.py to init.py in the genrule is getting clobbered because six is an existing dependency. A manual copy that you suggest fixes this for me as well.

[raulAtNines]

@laurit17 After trying several fixes for containerregistry and rules_docker, I just found the easiest workaround. At the top of the WORKSPACE for your project, add the same six fix they apply to the libraries:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

# Used by oauth2client
http_archive(
    name = "six",
    url = "https://pypi.python.org/packages/source/s/six/six-1.9.0.tar.gz",
    sha256 = "e24052411fc4fbd1f672635537c3fc2330d9481b18c0317695b46259512c91d5",
    strip_prefix = "six-1.9.0/",
    type = "tar.gz",
    build_file_content = """
# Rename six.py to __init__.py
genrule(
    name = "rename",
    srcs = ["six.py"],
    outs = ["__init__.py"],
    cmd = "cat $< >$@",
)
py_library(
    name = "six",
    srcs = [":__init__.py"],
    visibility = ["//visibility:public"],
)""",
)

That seems to have fixed it for me.

[raulAtNines]

The workaround is a copy of:
https://github.com/bazelbuild/rules_docker/blob/master/repositories/repositories.bzl#L161-L182
which comes from:
https://github.com/google/containerregistry/blob/master/def.bzl#L36-L56

[nlopezgi]

ugh, this seems to be a diamond dependency problem, which are not really easy to track down or resolve. Alternatively, you can try to make sure the call to @io_bazel_rules_docker//repositories:repositories.bzl is higher in your WORKSPACE than any other declarations that will load six. This is because, if you load six transitively before, the conditional in repositories.bzl will make it so that it does not get pulled again.
Not sure we can do anything in this repo to make this easier other than perhaps add a note to the README to warn about making sure the call to repositories is as high up as possible.

[raulAtNines]

Can confirm that adding:

git_repository(
    name = "containerregistry",
    commit = "da03b395ccdc4e149e34fbb540483efce962dc64",
    remote = "https://github.com/google/containerregistry",
)

load("@containerregistry//:def.bzl", cr_repositories = "repositories")
cr_repositories()

or

http_archive(
    name = "io_bazel_rules_docker",
    sha256 = "87fc6a2b128147a0a3039a2fd0b53cc1f2ed5adb8716f50756544a572999ae9a",
    strip_prefix = "rules_docker-0.8.1",
    urls = ["https://github.com/bazelbuild/rules_docker/archive/v0.8.1.tar.gz"],
  )

load(
    "@io_bazel_rules_docker//repositories:repositories.bzl",
    container_repositories = "repositories",
)
container_repositories()

to the top of our WORKSPACE also works.

[raulAtNines]

@nlopezgi It would be great to add that piece of info as a "known issue" to the readme. I had read about the diamond dependencies, but given the PY2 vs PY3 changes as well, it took a while to figure out what it was. Might save people some time.

Will prepare a PR.

[nlopezgi]

@raulAtNines that would be awesome, thanks!

[raulAtNines]

@nlopezgi Saw your approval, will sign the CLA shortly.
Thank you.

[raulAtNines]

For context, issue related to:
#55
#367
google/containerregistry#36

[prestonvanloon]

Traceback (most recent call last):
--
  | File "/root/.cache/bazel/_bazel_root/1178effa7ba5e3370fdb0c7fb51719e8/execroot/__main__/bazel-out/k8-py2-fastbuild/bin/validator/push_images.runfiles/containerregistry/tools/fast_pusher_.py", line 29, in <module>
  | from containerregistry.client import docker_creds
  | File "/root/.cache/bazel/_bazel_root/1178effa7ba5e3370fdb0c7fb51719e8/execroot/__main__/bazel-out/k8-py2-fastbuild/bin/validator/push_images.runfiles/containerregistry/client/__init__.py", line 19, in <module>
  | from containerregistry.client import docker_name_
  | File "/root/.cache/bazel/_bazel_root/1178effa7ba5e3370fdb0c7fb51719e8/execroot/__main__/bazel-out/k8-py2-fastbuild/bin/validator/push_images.runfiles/containerregistry/client/docker_name_.py", line 23, in <module>
  | import six.moves.urllib.parse
  | ImportError: No module named six.moves.urllib.parse

I'm having a similar issue and the order of WORKSPACE dependencies seems to have no effect. My import error is slightly different. Not sure what is the issue here.

Bazel 0.28.0.

[prestonvanloon]

Nevermind, had to clear the cache and/or disable remote caching...

[nlopezgi]

docs have been updated to reflect details about this issue. Closing.