Skip to content

"Failed to load plugin WhatIfToolPluginLoader" caused by protobuf compatibility error #6195

@nfelt

Description

@nfelt

The current release of the What-If Tool plugin includes generated Python protos that are incompatible with the 4.x protobuf runtime, leading to this error at startup when the virtual environment only has the 4.x protobuf runtime:

E0210 14:56:01.342828 139767216988224 application.py:125] Failed to load plugin WhatIfToolPluginLoader.load; ignoring it.
Traceback (most recent call last):
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard/backend/application.py", line 123, in TensorBoardWSGIApp
    plugin = loader.load(context)
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard_plugin_wit/wit_plugin_loader.py", line 57, in load
    from tensorboard_plugin_wit.wit_plugin import WhatIfToolPlugin
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard_plugin_wit/wit_plugin.py", line 40, in <module>
    from tensorboard_plugin_wit._utils import common_utils
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard_plugin_wit/_utils/common_utils.py", line 17, in <module>
    from tensorboard_plugin_wit._vendor.tensorflow_serving.apis import classification_pb2
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard_plugin_wit/_vendor/tensorflow_serving/apis/classification_pb2.py", line 15, in <module>
    from tensorboard_plugin_wit._vendor.tensorflow_serving.apis import input_pb2 as tensorflow__serving_dot_apis_dot_input__pb2
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard_plugin_wit/_vendor/tensorflow_serving/apis/input_pb2.py", line 37, in <module>
    _descriptor.FieldDescriptor(
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
TensorBoard 2.12.0a20230209 at http://xn--260a.mtv.corp.google.com:6006/ (Press CTRL+C to quit)

Note: this doesn't prevent TensorBoard overall from running, it only leads to the What-If Tool plugin being unavailable, but it's a large and scary error message that users are likely to notice.

We didn't notice this previously because TensorBoard itself had generated protos incompatible with the 4.x runtime (issue #5703), but that restriction has recently been lifted with #6147 and released in TensorBoard 2.12. For users who get tensorboard as a dependency of tensorflow, this won't yet affect them because TensorFlow also has required the 3.x protobuf runtime, but the soon-to-be-released TF 2.12 require the 4.x runtime, which will likely result in many more users noticing the error and not having a good workaround.

The solution is for the What-If Tool plugin to release a new version with protos generated with protoc 3.19 or higher.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions