ImportError: cannot import name '_message' from 'google.protobuf.pyext' (/usr/lib/python3.7/site-packages/google/protobuf/pyext/__init__.py)

[chroteus]

• TensorBoard version: tensorboard-1.12.0
• GNU/Linux, Manjaro
• Python version: 3.7.1

Issue: Just calling tensorboard fails to load. A bug with protobuf?

Traceback (most recent call last):
  File "/usr/bin/tensorboard", line 11, in <module>
    load_entry_point('tensorboard==1.12.0', 'console_scripts', 'tensorboard')()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 484, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2725, in load_entry_point
    return ep.load()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2343, in load
    return self.resolve()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2349, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3.7/site-packages/tensorboard/main.py", line 45, in <module>
    from tensorboard import default
  File "/usr/lib/python3.7/site-packages/tensorboard/default.py", line 34, in <module>
    import tensorflow as tf
  File "/usr/lib/python3.7/site-packages/tensorflow/__init__.py", line 22, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/usr/lib/python3.7/site-packages/tensorflow/python/__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/usr/lib/python3.7/site-packages/tensorflow/core/framework/graph_pb2.py", line 6, in <module>
    from google.protobuf import descriptor as _descriptor
  File "/usr/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 47, in <module>
    from google.protobuf.pyext import _message
ImportError: cannot import name '_message' from 'google.protobuf.pyext' (/usr/lib/python3.7/site-packages/google/protobuf/pyext/__init__.py)

[nfelt]

@chroteus It looks like you don't have the C++ implementation of protobuf available, and I guess protobuf doesn't check for that before attempting to use it. What version of python protobuf are you using and how did you install it (e.g. via pip)?

I just checked and the pip builds for python protobuf include the C++ implementation starting with 3.5.2, so if your version is below that could you try updating and see if that fixes it? If so we can update our setup.py dep to avoid this issue going forward.

[chroteus]

I have it installed it using Arch's package manager (Pacman).
Pacman pulls C++ implementation of protobuf as a dependency of python-protobuf anyway, so it doesn't seem to be the case.
Version - python-protobuf: 3.6.1-1

I will try uninstalling with Pacman and see if installing everything through pip will work.

EDIT: It seems the issue is with me using Python 3.7. I will downgrade and see if this is the case.
EDIT2: Downgrading messes up other packages that require Py3.7, so I guess we'll wait until Py3.7 is fully supported.

[nfelt]

Hmm, strange. Perhaps there's some way in which the pacman-installed python-protobuf C++ deps aren't being found appropriately under python 3.7.

Can you reproduce the issue with only protobuf? E.g. perhaps something like from google.protobuf.empty_pb2 import Empty; Empty.FromString(b'')?
If so then your best bet would probably be filing an issue with them.

[gariepyalex]

I have the same issue. from google.protobuf.empty_pb2 import Empty; Empty.FromString(b'') doesn't cause any problem.

[gariepyalex]

Found a solution here:
https://stackoverflow.com/a/50950265

export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp
sudo pip install --upgrade --force-reinstall protobuf

[nfelt]

@gariepyalex Thanks for the additional detail! I didn't have to set the env variable at installation time for pip packages on debian or mac. I had assumed the same would be true for windows but based on the StackOverflow question, perhaps not. I see here that maybe windows still omits the C++ dep:
protocolbuffers/protobuf#1332 (comment) I guess we'll need to follow up with them about that omission.

@chroteus I'm not really familiar at all with Pacman but is it possible the python-protobuf package there is not actually bundling/building the C++ extension code, even though it depends on the C++ protobuf package?

And I guess if the Empty test doesn't work it's perhaps better to directly check if from google.protobuf.pyext import _message succeeds or not.

[nfelt]

@chroteus FYI - I ran into this issue when running TensorBoard without the regular google protobuf pip package actually being available. I.e. python -c "import google.protobuf" didn't work at all in that virtual environment. I think the issue that TensorBoard's build bundles some amount of the protobuf runtime with it but not the C++ implementation. I was able to fix the issue just be doing pip install protobuf. So it's possible that your protobuf as provided by pacman isn't actually accessible from TensorBoard when it runs.

Note that this wouldn't apply to the problem as seen by @gariepyalex since in that case importing google.protobuf directly seems to work?

[getzze]

Like @chroteus said, it is an issue with the CPP implementation of protobuf not being compiled for python 3.7 yet: protocolbuffers/protobuf#5154
For Arch users (and others that need to use python3.7), we have to wait, installing from pip doesn't help.

[eli-schwartz]

https://bugs.archlinux.org/task/54959

Please don't break your system with sudo pip, by the way.

[showgood163]

I'm using arch linux, and this problem seems to start from tensorboard 1.11.0 and is still not solved by now.
My solution is that using downgrade(from AUR) to manually downgrade tensorboard to 1.10.0, which is the last working version.

[imyhxy]

I met this problem at tensorboard 1.12.0 and I solved it by downgrading tensorboard to 1.11.0.

[dniku]

The latest version that seems to work is tensorboard-1.11.0-2.

[azymohliad]

Another ArchLinux guy with the same issue. Fixed by updating python-protobuf from testing repo (released there on Dec 26th), just in case someone needs it urgently. But since it's already in testing the stable package should be available soon.

[samclane]

Found a solution here:
https://stackoverflow.com/a/50950265

export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp
sudo pip install --upgrade --force-reinstall protobuf

I just want to add, even if you're using a conda environment, you still have to do this through pip. I was on Windows, and ran: pip install --upgrade --force-reinstall protobuf to great success. Thanks again!

[Ekinkit]

I met the same problem and I failed to solve it. The anaconda prompt showed that my protobuf is 3.4.0 and my tensorflow is 1.12.0 tensorboard is 1.12.2. Is there any solution for me to solve it? Or can I find another way to figure out the average precision?(I am a beginner)