ImportError: symbol not found in flat namespace (protobuf 3.20.2) #10571

pietrodn · 2022-09-14T19:26:28Z

What version of protobuf and what language are you using?
Python, protobuf==3.20.2

What operating system (Linux, Windows, ...) and version?
macOS 12.6

What runtime / compiler are you using (e.g., python version or gcc version)
Python 3.10.6

What did you do?

python3 -m venv venv
. venv/bin/activate
pip install protobuf==3.20.2
python3 -c "import google.protobuf.descriptor"

What did you expect to see
Nothing (a successful import)

What did you see instead?

>>> import google.protobuf.descriptor
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".venv/lib/python3.10/site-packages/google/protobuf/descriptor.py", line 47, in <module>
    from google.protobuf.pyext import _message
ImportError: dlopen(.venv/lib/python3.10/site-packages/google/protobuf/pyext/_message.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace (__ZN6google8protobuf15FieldDescriptor12TypeOnceInitEPKS1_)

FYI a similar error is happening inside Google's google-cloud-bigquery library.

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

Anything else we should know about your project / environment

The issue does not happen in a Linux environment (our CI).
The issue does not happen with protobuf==3.20.1.
The issue does not happen on Python 3.8.

The text was updated successfully, but these errors were encountered:

jcrm1 · 2022-09-15T05:09:19Z

Also getting this error (still on macOS 12.6, Python 3.10.6)

madsjk816 · 2022-09-15T09:09:29Z

+1 here, macOS 12.6, Python 3.10.6, protobuf==3.20.2 and Mac M1

miohtama · 2022-09-15T11:11:50Z

Same: macOS 12.6, Python 3.10.6, protobuf==3.20.2 and Mac M1.

For me, I am not using even using Protobuf, but one of the dependencies imports it on a startup and this side effect causes the codebase no longer run:

    from ethpm.dependencies import (
  File "/Users/moo/Library/Caches/pypoetry/virtualenvs/backend-WPuQnUi4-py3.10/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 168, in exec_module
    exec(co, module.__dict__)
  File "/Users/moo/Library/Caches/pypoetry/virtualenvs/backend-WPuQnUi4-py3.10/lib/python3.10/site-packages/ethpm/dependencies.py", line 7, in <module>
    from ethpm.validation.package import (
  File "/Users/moo/Library/Caches/pypoetry/virtualenvs/backend-WPuQnUi4-py3.10/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 168, in exec_module
    exec(co, module.__dict__)
  File "/Users/moo/Library/Caches/pypoetry/virtualenvs/backend-WPuQnUi4-py3.10/lib/python3.10/site-packages/ethpm/validation/package.py", line 11, in <module>
    from ethpm._utils.ipfs import (
  File "/Users/moo/Library/Caches/pypoetry/virtualenvs/backend-WPuQnUi4-py3.10/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 168, in exec_module
    exec(co, module.__dict__)
  File "/Users/moo/Library/Caches/pypoetry/virtualenvs/backend-WPuQnUi4-py3.10/lib/python3.10/site-packages/ethpm/_utils/ipfs.py", line 18, in <module>
    from google.protobuf.descriptor import (
  File "/Users/moo/Library/Caches/pypoetry/virtualenvs/backend-WPuQnUi4-py3.10/lib/python3.10/site-packages/google/protobuf/descriptor.py", line 47, in <module>
    from google.protobuf.pyext import _message

miohtama · 2022-09-15T11:25:02Z

Confirming downgrade works:

  • Updating protobuf (3.20.2 -> 3.20.1)

cinnil · 2022-09-15T14:52:04Z

Had this same issue pop up while installing lstein/stable-diffusion and downgrading solved it as well. Thanks!

Confirming downgrade works:
  • Updating protobuf (3.20.2 -> 3.20.1)

AlexSchultz-clumio · 2022-09-15T19:05:42Z

3.19.5 also suffers from the same issue. Downgrading to 3.19.4 works.

nikiz24 · 2022-09-16T02:59:40Z

in my case
brew install protobuf
brew link --overwrite protobuf
works. Others like downgrade pip protobuf does not work for me.

nikiz24 · 2022-09-16T02:59:49Z

in my case
brew install protobuf
brew link --overwrite protobuf
works. Others like downgrade pip protobuf does not work for me.

scomma · 2022-09-16T08:39:12Z

3.19.5 also suffers from the same issue. Downgrading to 3.19.4 works.

Confirmed fix, thanks! Came here trying to install stable diffuser.

bh2smith · 2022-09-16T11:49:16Z

Issue also doesn't happen on python 3.9!

Ported · 2022-09-16T14:45:29Z

Running "pip install protobuf==3.19.4" to downgrade also worked for me.

- add url init images - don't install protobuf versions that are causing problems #7 protocolbuffers/protobuf#10571

pro0517 · 2022-09-21T06:25:31Z

+1 here, macOS 11.2, Python 3.10.7, protobuf==3.20.2 and Mac M1

lightnessofbein · 2022-09-22T09:13:56Z

I used to have troubles with protobuf==3.20.2 (ImportError cause by some symbols) right after import feast
Downgrading to 3.20.1 solves the problem

Roymprog · 2022-09-22T12:46:39Z

I do experience the same problem with Python version 3.8.13 using poetry. Perhaps it's related to this issue.
Neither version 3.20.1 nor 3.19.4 seems to work.

I have found that protobuf-3.20.1-py2.py3-none-any.whl fails, and protobuf-3.20.1-cp38-cp38-macosx_10_9_x86_64.whl works.

EDIT:
Solved my issue by being explicit about how to build protobuf on M1 Mac:
pip install protobuf==3.20 --install-option="--cpp_implementation"

pietrodn · 2022-10-03T14:09:06Z

It appears that the same bug was introduced in parallel in 3.19.5 and 3.20.2, most likely as a result of backporting some bug fix to the earlier minor revision.

deannagarcia · 2022-10-03T20:16:08Z

Sorry about this issue. The problem is that the 3.19 and 3.20 branches only used the pure python implementation of protobuf. The newest update has added an incomplete C++ implementation for M1 binaries which has caused the crash.

To work around, you can set the environment variable PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python.

We are working on a fix now, but in order to get the most efficient python implementation we would recommend you upgrade to protobuf 4.21.7.

deannagarcia · 2022-10-03T22:49:30Z

I believe this issue should be fixed for 3.19.5, 3.19.6, 3.20.2, and 3.20.3. Please let me know if you are still experiencing issues.

The fix will force python 3.10 osx users to use pure python. If performance is important to you, we highly recommend you upgrade to 4.21.7.

pietrodn · 2022-10-04T07:44:04Z

I confirm the issue is fixed for 3.19.5, 3.19.6, 3.20.2, and 3.20.3 on Python 3.10.6, macOS 12.6, M1 Pro processor. 😄

Thank you!

- Start from ``3.10.2`` since a security vulnerability was fixed there: GHSA-8gq9-2x98-w8hf - Discussion here: protocolbuffers/protobuf#10571

- Start from ``3.20.2`` since a security vulnerability was fixed there: GHSA-8gq9-2x98-w8hf - Discussion here: protocolbuffers/protobuf#10571

nmariz · 2022-10-10T10:40:32Z

I confirm the issue is fixed for 3.19.5, 3.19.6, 3.20.2, and 3.20.3 on Python 3.10.6, macOS 12.6, M1 Pro processor. 😄

No it's not.

$ python --version
Python 3.10.3

$ python -c "import google.protobuf; print(google.protobuf.__version__)"
3.20.3

$ python -c "import google.protobuf.descriptor"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/nmariz/tmp/venv/lib/python3.10/site-packages/google/protobuf/descriptor.py", line 47, in <module>
    from google.protobuf.pyext import _message
ImportError: dlopen(/Users/nmariz/tmp/venv/lib/python3.10/site-packages/google/protobuf/pyext/_message.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace (__ZN6google8protobuf15FieldDescriptor12TypeOnceInitEPKS1_)

And upgrading to 4.21.7 is not a workable solution, since the stubs are incompatible with 3.x. As well as the environment variable, specially for libraries that use Protobuf. I don't get it why the bug is closed and not fixed. Basically all macOS users will have to use a release with a known security issue (in the 3.20 series).

pietrodn · 2022-10-10T12:38:28Z

@nmariz, somehow the import works perfectly on my Mac with Python 3.10.6 and Protobuf 3.20.3. Very weird. 🤔

deannagarcia · 2022-10-10T17:08:41Z

Sorry you're still seeing this issue, I thought it was resolved.

I was able to reproduce with the steps in the original bug and after my change the error went away. When you use those steps are you still seeing the error?

Can you provide reproduction steps that include installing protobuf? Have you uninstalled/reinstalled?

nmariz · 2022-10-10T17:53:35Z

Thanks for considering my comment, but please ignore it.
Indeed it doesn't raise the error now, because I was using a binary version with the C extension, that I've installed with pip. I think you've yanked that package from PyPI and it's not available anymore, so for macOS ARM64 only the pure Python is available.

* update to get list of tasks from parameters * upgrade protobuf protocolbuffers/protobuf#10571 * remove write_parameters_file. We are using metadata * remove json * update launch for testing * update flat payment parameters * update generic rule * add pydash * update main run * update payment tree to add column * update flat payment * get scheduler to flatten explanation block into parameters * rename run_reward_program to run_task * clean * fix to have default for explanation_data_arr * fix rollover test * fix safe ownership test * fix retro airdrop test * fix staking test * fix flat payment test * fix add metadata test * remove unused import lint * update input parameters * update staking * remove explanation_id argument from as_arrow * make explanationData part of payment * remove explanation id and explanation block from rule

* update to get list of tasks from parameters * upgrade protobuf protocolbuffers/protobuf#10571 * remove write_parameters_file. We are using metadata * remove json * update launch for testing * update flat payment parameters * update generic rule * add pydash * update main run * update payment tree to add column * update flat payment * get scheduler to flatten explanation block into parameters * rename run_reward_program to run_task * clean * fix to have default for explanation_data_arr * fix rollover test * fix safe ownership test * fix retro airdrop test * fix staking test * fix flat payment test * fix add metadata test * remove unused import lint * update input parameters * update staking * remove explanation_id argument from as_arrow * make explanationData part of payment * remove explanation id and explanation block from rule * update generic rule * add rollover_amount to explanation * fix rollover test * remove group_by import * add explanation for staking * fix black * fix staking rule * fix merge * remove group_by function * Update indexer and api to use explanation (#3352) * update generic rule * update idnexer * update flat_drop script * update api * add laucnh files for debugging * update flat payment params * fix * remove setuptools * just trigger * update parquet files for test * update staking params * add deafaults for files with no explanationId or explanationData * fix reward api project toml * add name and version fro indexer * fix merge * remove itertools import

marinagalenko · 2022-11-11T10:30:53Z

Hello!
Not sure if I should comment here since the issue is closed, but I'm experiencing the same issue.

sw_vers
ProductName:		macOS
ProductVersion:		13.0.1
BuildVersion:		22A400

python --version
Python 3.10.7

python -c "import google.protobuf; print(google.protobuf.__version__)"
3.20.3

python -c "import google.protobuf.descriptor"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/marina.galenko/Documents/GSL/hudl-videospa/tests/ui/venv/lib/python3.10/site-packages/google/protobuf/descriptor.py", line 47, in <module>
    from google.protobuf.pyext import _message
ImportError: dlopen(/Users/marina.galenko/Documents/GSL/hudl-videospa/tests/ui/venv/lib/python3.10/site-packages/google/protobuf/pyext/_message.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '__ZN6google8protobuf15FieldDescriptor12TypeOnceInitEPKS1_'

fabiogra · 2023-05-04T09:53:32Z

I solved downgrading Python to 3.9.16 (Apple M1 macOS 12.6)

Python 3.9.16 (main, Mar  8 2023, 04:29:24) 
[Clang 14.0.6 ] :: Anaconda, Inc. on darwin

>>> import google.protobuf; print(google.protobuf.__version__)
3.20.3
>>> import google.protobuf.descriptor
>>>

- add url init images - don't install protobuf versions that are causing problems brycedrennan/imaginAIry#7 protocolbuffers/protobuf#10571

pietrodn added the untriaged auto added to all issues by default when created. label Sep 14, 2022

pietrodn changed the title ~~ImportError~~ ImportError: symbol not found in flat namespace (protobuf 3.20.2) Sep 14, 2022

fowles assigned deannagarcia Sep 15, 2022

fowles added python packaging & distribution and removed untriaged auto added to all issues by default when created. labels Sep 15, 2022

ipenywis mentioned this issue Sep 15, 2022

fix for dlopen "symbol not found in flat namespace" error invoke-ai/InvokeAI#580

Closed

abij mentioned this issue Sep 16, 2022

Requirements to make it work on M1 brycedrennan/imaginAIry#7

Closed

ejguan mentioned this issue Sep 16, 2022

Failing CI tests on MacOS with Python 3.10 caused by protobuf pytorch/data#775

Closed

brycedrennan added a commit to brycedrennan/imaginAIry that referenced this issue Sep 16, 2022

version 1.2.0

33f1e2a

- add url init images - don't install protobuf versions that are causing problems #7 protocolbuffers/protobuf#10571

burakkurkcu mentioned this issue Sep 19, 2022

No module named 'transformers' on initial Arm MacOS setup invoke-ai/InvokeAI#650

Closed

harskish mentioned this issue Sep 19, 2022

Run-time error on MacOS 12.6: symbol not found in flat namespace (__ZN6google8protobuf15FieldDescriptor12TypeOnceInitEPKS1_) harskish/sdui#5

Closed

Modjular mentioned this issue Sep 19, 2022

Protobuf 3.20.2 crashes grpc server. ImportError: symbol not found in flat namespace dagster-io/dagster#9728

Closed

lukasmasuch mentioned this issue Sep 21, 2022

Ignore broken protobuf version streamlit/streamlit#5396

Merged

4 tasks

jdries3 mentioned this issue Sep 21, 2022

Codeformer face restoration fails - Placeholder storage has not been allocated on MPS device invoke-ai/InvokeAI#713

Closed

ghost mentioned this issue Sep 22, 2022

fix: pin protobuf version 3.20.1 ApeWorX/ape#1071

Closed

3 tasks

pietrodn mentioned this issue Oct 4, 2022

protobuf dependency compatibility ethereum/web3.py#2654

Closed

deannagarcia closed this as completed Oct 4, 2022

fselmo added a commit to fselmo/web3.py that referenced this issue Oct 4, 2022

Unpin protobuf. The bug has been fixed on their end.

ef84f3d

- Start from ``3.10.2`` since a security vulnerability was fixed there: GHSA-8gq9-2x98-w8hf - Discussion here: protocolbuffers/protobuf#10571

fselmo mentioned this issue Oct 4, 2022

[WIP] Unpin protobuf. The bug has been fixed on their end. ethereum/web3.py#2664

Closed

1 task

fselmo added a commit to fselmo/web3.py that referenced this issue Oct 4, 2022

Unpin protobuf. The bug has been fixed on their end.

2ff2910

- Start from ``3.20.2`` since a security vulnerability was fixed there: GHSA-8gq9-2x98-w8hf - Discussion here: protocolbuffers/protobuf#10571

lukasmasuch mentioned this issue Oct 4, 2022

Remove protobuf pinning streamlit/streamlit#5478

Merged

4 tasks

deannagarcia reopened this Oct 10, 2022

deannagarcia closed this as completed Oct 10, 2022

tintinthong added a commit to cardstack/cardstack that referenced this issue Oct 14, 2022

upgrade protobuf protocolbuffers/protobuf#10571

d470c47

tintinthong added a commit to cardstack/cardstack that referenced this issue Oct 25, 2022

upgrade protobuf protocolbuffers/protobuf#10571

e3a2bcc

leplatrem mentioned this issue Nov 29, 2022

Upgrade protobuf to 4.21.9 mozilla-services/telescope#1133

Merged

SoftwareLiteracy mentioned this issue Mar 5, 2023

pyEDM install error 2023 Apple M2 Pro running Ventura SugiharaLab/pyEDM#36

Closed

xxchan mentioned this issue Mar 17, 2023

Cannot run risingwave binary on Mac OS M1 (when linked by zld) risingwavelabs/risingwave#8608

Closed

flexatone mentioned this issue May 3, 2023

symbol not found in flat namespace '_npy_double_to_half' static-frame/arraykit#108

Closed

mattstern31 added a commit to mattstern31/imagin-AIry-Python that referenced this issue Nov 11, 2023

version 1.2.0

8e9520a

- add url init images - don't install protobuf versions that are causing problems brycedrennan/imaginAIry#7 protocolbuffers/protobuf#10571

penberg mentioned this issue Mar 25, 2024

symbol not found in found in namespace '_sqlite3_wal_manager' tursodatabase/libsql-experimental-python#47

Closed

teaguesterling mentioned this issue May 23, 2024

Update py-pyspark and py-py4j spack/spack#44263

Merged

superlucky19971023 added a commit to superlucky19971023/imaginAIry that referenced this issue Aug 18, 2024

version 1.2.0

c618d80

- add url init images - don't install protobuf versions that are causing problems brycedrennan/imaginAIry#7 protocolbuffers/protobuf#10571

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ImportError: symbol not found in flat namespace (protobuf 3.20.2) #10571

ImportError: symbol not found in flat namespace (protobuf 3.20.2) #10571

pietrodn commented Sep 14, 2022 •

edited

Loading

jcrm1 commented Sep 15, 2022

madsjk816 commented Sep 15, 2022

miohtama commented Sep 15, 2022 •

edited

Loading

miohtama commented Sep 15, 2022

cinnil commented Sep 15, 2022

AlexSchultz-clumio commented Sep 15, 2022

nikiz24 commented Sep 16, 2022

nikiz24 commented Sep 16, 2022

scomma commented Sep 16, 2022

bh2smith commented Sep 16, 2022

Ported commented Sep 16, 2022

pro0517 commented Sep 21, 2022

lightnessofbein commented Sep 22, 2022

Roymprog commented Sep 22, 2022 •

edited

Loading

pietrodn commented Oct 3, 2022

deannagarcia commented Oct 3, 2022

deannagarcia commented Oct 3, 2022

pietrodn commented Oct 4, 2022

nmariz commented Oct 10, 2022

pietrodn commented Oct 10, 2022

deannagarcia commented Oct 10, 2022

nmariz commented Oct 10, 2022 •

edited

Loading

marinagalenko commented Nov 11, 2022

fabiogra commented May 4, 2023

ImportError: symbol not found in flat namespace (protobuf 3.20.2) #10571

ImportError: symbol not found in flat namespace (protobuf 3.20.2) #10571

Comments

pietrodn commented Sep 14, 2022 • edited Loading

jcrm1 commented Sep 15, 2022

madsjk816 commented Sep 15, 2022

miohtama commented Sep 15, 2022 • edited Loading

miohtama commented Sep 15, 2022

cinnil commented Sep 15, 2022

AlexSchultz-clumio commented Sep 15, 2022

nikiz24 commented Sep 16, 2022

nikiz24 commented Sep 16, 2022

scomma commented Sep 16, 2022

bh2smith commented Sep 16, 2022

Ported commented Sep 16, 2022

pro0517 commented Sep 21, 2022

lightnessofbein commented Sep 22, 2022

Roymprog commented Sep 22, 2022 • edited Loading

pietrodn commented Oct 3, 2022

deannagarcia commented Oct 3, 2022

deannagarcia commented Oct 3, 2022

pietrodn commented Oct 4, 2022

nmariz commented Oct 10, 2022

pietrodn commented Oct 10, 2022

deannagarcia commented Oct 10, 2022

nmariz commented Oct 10, 2022 • edited Loading

marinagalenko commented Nov 11, 2022

fabiogra commented May 4, 2023

pietrodn commented Sep 14, 2022 •

edited

Loading

miohtama commented Sep 15, 2022 •

edited

Loading

Roymprog commented Sep 22, 2022 •

edited

Loading

nmariz commented Oct 10, 2022 •

edited

Loading