installation failed on mac osx 10.15 #114

PiPinoccio · 2020-04-25T14:52:26Z

Hello,
I know that there are certain issues with the Mac OSX and their version of the GCC compiler. I did install the Gnu GCC, but all the recommended methods of convincing OSX to use the Gnu compiler have failed.

I tried to install plyvel with pip and got fail messages. This is what I get - at least the essential bit.

Complete output (16 lines): running install running build running build_py creating build creating build/lib.macosx-10.6-intel-3.7 creating build/lib.macosx-10.6-intel-3.7/plyvel copying plyvel/_version.py -> build/lib.macosx-10.6-intel-3.7/plyvel copying plyvel/__init__.py -> build/lib.macosx-10.6-intel-3.7/plyvel running build_ext building 'plyvel._plyvel' extension creating build/temp.macosx-10.6-intel-3.7 creating build/temp.macosx-10.6-intel-3.7/plyvel gcc -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c plyvel/_plyvel.cpp -o build/temp.macosx-10.6-intel-3.7/plyvel/_plyvel.o -Wall -g -x c++ -std=c++11 -stdlib=libc++ clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) error: command 'gcc' failed with exit status 1 ----------------------------------------

I obviously have an OSX later than 10.7, but for some reason this is not recognised.
I would be very grateful for any help concerning this issue.

Best
Pia

The text was updated successfully, but these errors were encountered:

wbolster · 2020-04-26T16:20:52Z

i don't use osx myself, sorry 😕

that said, there are a few seemingly similar (closed) issues in the issue tracker. maybe you can have a look, and if that doesn't help, gently ping some more knowledgeable people in this issue.

bocheng0000 · 2020-05-08T10:38:29Z

I met one similar issue and have solved it by installing leveldb first.

brew install leveldb
pip install plyvel

bagonyi · 2021-03-02T18:48:43Z

In my case I was not able to install plyvel in a virtual environment that was created from the macOS provided Python 3.8 installation. pip install plyvel was giving me all sorts of compilation errors. As soon as I told virtualenv to create the virtual environment from the brew provided Python 3.8, plyvel installed without any issues.

Try:

brew install leveldb
brew install python@3.8
virtualenv --python /usr/local/opt/python@3.8/bin/python3 venv
source venv/bin/activate
pip install plyvel

Update:

Spoke too soon. Installation went fine, however when I try to run a Python script that uses plyvel I get: Symbol not found: __ZTIN7leveldb10ComparatorE...

sr-gi · 2021-03-10T18:05:14Z

Update:

Spoke too soon. Installation went fine, however when I try to run a Python script that uses plyvel I get: Symbol not found: __ZTIN7leveldb10ComparatorE...

I'm getting the exact same error.

To give a little more context:

leveldb=1.23
plyvel=1.30

Tried both Python3.8 and Python3.9.

All worked well before updating, and leveldb released a new version 15 days ago, so I'm guessing there's gonna be something there, but cannot install 1.22 from brew again so I've been unable to test it :(

sr-gi · 2021-03-10T19:59:37Z

Got it to work by reinstalling leveldb 1.22, so I can confirm the issue is with the latest version.

wbolster · 2021-03-11T08:51:58Z

did you recompile plyvel after upgrading leveldb? if not, i think you should.

sr-gi · 2021-03-11T11:17:50Z

I did after upgrading / downgrading. It only work aster downgrading and recompiling.

wbolster · 2021-03-11T21:11:28Z

are you sure it recompiled (and didn't use some cached wheel somewhere on your disk)?

sr-gi · 2021-03-15T11:20:34Z

@wbolster looks like that was the issue indeed, it was building using and old cached version.

Sorry for the unnecessary noise.

rickwierenga · 2021-03-21T14:19:52Z

@sr-gi How did you install leveldb 1.22? It seems brew no longer supports it.

sr-gi · 2021-03-21T19:03:28Z

@rickwierenga I had to modify the formula myself to point to the proper version. You can get historical versions by browsing homebrew's GitHub.

CypherGrue · 2021-03-26T07:55:02Z

Seems to me that a leveldb 1.23 change disabled RTTI, and typeinfo symbols are missing from the built library, whereas they were present in leveldb 1.22.

See related issue google/leveldb#731

bagonyi · 2021-03-30T05:09:56Z

Managed to get it to work as well with leveldb version 1.22.

In order to get leveldb 1.22 installed using brew, first I had to create my own Homebrew tap. To get yours created, you can follow the instructions I put in my own Homebrew formulae repo: https://github.com/bagonyi/homebrew-formulae

Once leveldb with the correct version was installed, I uninstalled plyvel and reinstalled it with caching disabled:

pip uninstall plyvel
pip install plyvel --no-cache-dir

CypherGrue · 2021-03-30T05:21:28Z

Tested that a custom built leveldb 1.23, with the following line removed from its CMakeLists.txt, works fine.

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti")

maziyarpanahi · 2021-04-09T11:29:47Z

Thanks @bagonyi
Your 1.22 leveldb with the following worked for me on Big Sur:

pip uninstall plyvel
CFLAGS='-mmacosx-version-min=11.2 -stdlib=libc++ -std=c++11' pip install plyvel --upgrade

liviaerxin · 2021-05-06T06:22:23Z

Update:
Spoke too soon. Installation went fine, however when I try to run a Python script that uses plyvel I get: Symbol not found: __ZTIN7leveldb10ComparatorE...

I'm getting the exact same error.

To give a little more context:

leveldb=1.23
plyvel=1.30

Tried both Python3.8 and Python3.9.

All worked well before updating, and leveldb released a new version 15 days ago, so I'm guessing there's gonna be something there, but cannot install 1.22 from brew again so I've been unable to test it :(

I also encounter this error on OSX when compiling a static leveldb library libleveldb.a and building plyvel wheel with libleveldb.a. As @CypherGrue said in leveldb 1.23 diff with 1.22, it disables RTTI by add flag -fno-rtti. So when linking a static library libleveldb.a which did not have RTTI, I build _plyvel.cpython-37m-darwin.so also with no RTTI by add same cflags in setup.py.

...
extra_compile_args = ['-Wall', '-g', '-x', 'c++', '-std=c++11', '-fno-rtti']
...

After that, this problem was fixed.

python3 -c "import plyvel; print(plyvel.__leveldb_version__); print(plyvel.__version__)"
1.23
1.3.0
(venv)

In addition, I also built plyvel on Windows. It is fine for msvc compiler when mixing RTTI and no RTTI, so you do not need add /GR- (RTTI disabled) on Windows

agatti · 2021-05-06T16:41:01Z

Seems like this also occurs when using MacPorts, although the error is different:

      /usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -I/Users/agatti/Library/Caches/pypoetry/virtualenvs/hs-workflows-plgAVo18-py3.8/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c plyvel/_plyvel.cpp -o build/temp.macosx-10.15-x86_64-3.8/plyvel/_plyvel.o -Wall -g -x c++ -std=c++11 -stdlib=libc++
      plyvel/_plyvel.cpp:632:10: fatal error: 'leveldb/db.h' file not found
      #include "leveldb/db.h"
               ^~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit status 1

Both Python 3.8 and its virtualenv package come from MacPorts so it should pick up the proper include directory (/opt/local/include) when building the package.

…rg/project/plyvel-wheels/#files - Which fixes this problem: wbolster/plyvel#114 for details on "Symbol not found: __ZTIN7leveldb10ComparatorE" error - By including a statically linked leveldb v 1.22 in the wheel (for Mac). - This also saves the user from needing to perform a "brew install leveldb" step and is therefore an essential part of making the SDK a one-click-installation...

ibnachraf · 2021-09-13T18:08:51Z

Hii, just installed leveldb@1.22 following @bagonyi tips and works fine for me

Managed to get it to work as well with leveldb version 1.22.

In order to get leveldb 1.22 installed using brew, first I had to create my own Homebrew tap. To get yours created, you can follow the instructions I put in my own Homebrew formulae repo: https://github.com/bagonyi/homebrew-formulae

Once leveldb with the correct version was installed, I uninstalled plyvel and reinstalled it with caching disabled:
pip uninstall plyvel
pip install plyvel --no-cache-dir

YSaxon · 2022-02-28T17:13:10Z

plyvel/_plyvel.cpp:703:10: fatal error: 'leveldb/db.h' file not found
      #include "leveldb/db.h"
               ^~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

I'm also getting this error on trying to pip install plyvel or install it from source, with leveldb installed through homebrew, on an M1 (arm) mac

YSaxon · 2022-02-28T17:15:57Z

Just solved this by prefixing either the make or the pip install with CFLAGS="-I/opt/homebrew/include -L/opt/homebrew/lib", as suggested by the following stack exchange answer.
https://stackoverflow.com/a/70375604

3cola · 2022-03-17T11:43:48Z

I don't know if that help, but I succeeded to build it on freebsd with this command:
CFLAGS='-g -stdlib=libc++ -std=c++11 -fno-rtti' pip install --force-reinstall --global-option="build_ext" --global-option="-I/usr/local/include" --global-option="-L/usr/local/lib" plyvel

trungtin · 2022-06-22T07:50:15Z

CFLAGS='-g -stdlib=libc++ -std=c++11 -fno-rtti' pip install --force-reinstall --global-option="build_ext" --global-option="-I/usr/local/include" --global-option="-L/usr/local/lib" plyvel

M1 Mac Monterey. Using this + build leveldb from source @ 1.22.0 help plyvel find leveldb. But it still cannot find snappy. Have tried install snappy from source or brew

ImportError: dlopen(/opt/homebrew/Caskroom/miniforge/base/envs/scraper/lib/python3.10/site-packages/plyvel/_plyvel.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '__ZN6snappy11RawCompressEPKcmPcPm'

arlaneenalra · 2022-09-27T23:07:02Z

This worked for me with Monterey and homebrew:

CFLAGS="-I/opt/homebrew/include/ -L/opt/homebrew/lib/ -fno-rtti" pip install --force-reinstall --global-option="build_ext" plyvel

I'm on an M1 machine so you may have to amend the -I and -L argments.

Siemingfong · 2023-01-24T11:05:26Z

@bocheng0000 Thanks! install leveldb before plyvel is works for me.

nicolasff · 2023-06-18T03:01:33Z

The command given above might not work for everyone, if for whatever reason the Homebrew symlinks are not under /opt/homebrew.

You can get the install directory of any Homebrew package with brew --prefix, so adapting the command to use this dynamic value gives:

CFLAGS="-I$(brew --prefix leveldb)/include/ -L$(brew --prefix leveldb)/lib/ -fno-rtti" pip install --force-reinstall --global-option="build_ext" plyvel

Verify that the library was linked without issues by simply loading the module:

python -c 'import plyvel'

catap · 2023-09-06T09:10:36Z

Another approach is switch to MacPorts where I keep RTTI at LevelDB as a variant and by default it builds with it. So, plyvel works out of the box :)

Pandemolde · 2023-10-16T17:07:57Z

plyvel/_plyvel.cpp:703:10: fatal error: 'leveldb/db.h' file not found
      #include "leveldb/db.h"
               ^~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
I'm also getting this error on trying to pip install plyvel or install it from source, with leveldb installed through homebrew, on an M1 (arm) mac

Also happening from M2 (arm) mac... I already tried with leveldb@1.22 version and all flags specified in this thread.

timkpaine · 2024-01-06T14:40:30Z

This worked for me on M1 / 14.3 with simple brew install leveldb

CFLAGS="-I/opt/homebrew/include -w" LDFLAGS="-L/opt/homebrew/lib" pip install plyvel

lehrblogger · 2024-01-18T19:00:33Z

CFLAGS='-g -stdlib=libc++ -std=c++11 -fno-rtti' pip install --force-reinstall --global-option="build_ext" --global-option="-I/usr/local/include" --global-option="-L/usr/local/lib" plyvel

M1 Mac Monterey. Using this + build leveldb from source @ 1.22.0 help plyvel find leveldb. But it still cannot find snappy. Have tried install snappy from source or brew
ImportError: dlopen(/opt/homebrew/Caskroom/miniforge/base/envs/scraper/lib/python3.10/site-packages/plyvel/_plyvel.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '__ZN6snappy11RawCompressEPKcmPcPm'

I was running into the same issue as @trungtin, and solved it by building snappy and then leveldb from source with cmake -DBUILD_SHARED_LIBS=ON. I also had to add export DYLD_LIBRARY_PATH="$DYLD_LIBRARY_PATH:/usr/local/lib/" to make sure pylvel could find them.

Thanks for all the suggestions in this thread!

YuraAka · 2024-01-25T08:58:11Z

This works for me on MacOs Sonoma M2 Pro

CFLAGS="-I$(brew --prefix leveldb)/include/ -L$(brew --prefix leveldb)/lib/ -fno-rtti" pip install --force-reinstall --config-settings='--global-option="build_ext"' --no-cache-dir plyvel

markoelez · 2024-01-31T22:49:46Z

This works for me on MacOs Sonoma M2 Pro

CFLAGS="-I$(brew --prefix leveldb)/include/ -L$(brew --prefix leveldb)/lib/ -fno-rtti" pip install --force-reinstall --config-settings='--global-option="build_ext"' --no-cache-dir plyvel

This is the only solution that worked for me on an M3 Max running Sonoma 14.2.1.

xavi-clovr · 2024-02-01T16:58:55Z

This works for me on MacOs Sonoma M2 Pro

CFLAGS="-I$(brew --prefix leveldb)/include/ -L$(brew --prefix leveldb)/lib/ -fno-rtti" pip install --force-reinstall --config-settings='--global-option="build_ext"' --no-cache-dir plyvel

Same here! It worked to me with same proc! :)

jonathanykh · 2024-02-12T07:48:07Z

This works for me on MacOs Sonoma M2 Pro

CFLAGS="-I$(brew --prefix leveldb)/include/ -L$(brew --prefix leveldb)/lib/ -fno-rtti" pip install --force-reinstall --config-settings='--global-option="build_ext"' --no-cache-dir plyvel

works for me on M2 Ventura 13.6
thank you.

Traceback (most recent call last): File "/opt/venv/bin/pyrobud", line 5, in <module> from pyrobud.main import main File "/opt/venv/lib/python3.12/site-packages/pyrobud/main.py", line 4, in <module> from . import DEFAULT_CONFIG_PATH, __description__, launch, logs File "/opt/venv/lib/python3.12/site-packages/pyrobud/launch.py", line 9, in <module> from . import DEFAULT_CONFIG_PATH, util File "/opt/venv/lib/python3.12/site-packages/pyrobud/util/__init__.py", line 1, in <module> from . import ( File "/opt/venv/lib/python3.12/site-packages/pyrobud/util/config.py", line 11, in <module> from .config_db_migrator import upgrade_v3 File "/opt/venv/lib/python3.12/site-packages/pyrobud/util/config_db_migrator.py", line 4, in <module> import plyvel File "/opt/venv/lib/python3.12/site-packages/plyvel/__init__.py", line 6, in <module> from ._plyvel import ( # noqa ImportError: Error relocating /opt/venv/lib/python3.12/site-packages/plyvel/_plyvel.cpython-312-x86_64-linux-musl.so: _ZTIN7leveldb10ComparatorE: symbol not found * Ref: wbolster/plyvel#114 (comment) Change-Id: I1882a7bedbabed34a705dae385b884e754f01b19

mnba · 2024-06-15T16:57:22Z

All upper receipts didn't work for me, so I have found my solution:

brew install leveldb

CFLAGS="-I/usr/local/Cellar/leveldb/1.23_1/include -fno-rtti" LDFLAGS="-L/usr/local/Cellar/leveldb/1.23_1/lib"  pip3 install plyvel --no-cache-dir

dhdaines · 2024-09-10T12:13:44Z

None of the above commands work for me (on a venerable x86-64 MacBook Pro with MacOS 13.6 and (the important part) Python 3.12.1). The CFLAGS are not being passed into the build process (you can verify this by adding -v to the pip command line) so -fno-rtti, which is the important thing (to turn off some bogus C++ feature that nobody in their right mind uses anyway), doesn't get picked up in linking. The simplest solution is just to download the plyvel source code, make this change:

diff --git a/setup.py b/setup.py
index cdebab5..33dc75b 100644
--- a/setup.py
+++ b/setup.py
@@ -14,7 +14,7 @@ def get_file_contents(filename):
         return fp.read()
 
 
-extra_compile_args = ['-Wall', '-g', '-x', 'c++', '-std=c++11']
+extra_compile_args = ['-Wall', '-g', '-x', 'c++', '-std=c++11', '-fno-rtti']
 
 if platform.system() == 'Darwin':
     extra_compile_args += ['-stdlib=libc++']

then compile/install, no extra flags needed in my case:

pip install -r requirements-dev.txt # needed because see below
make  # needed because see below
pip install --force-reinstall .

Of course, plyvel has some ancient setup.py that doesn't even specify build dependencies or run Cython for you, so don't try to do like every other python package written since 2016 and just pip install . ... you have to run make...

dhdaines · 2024-09-10T13:14:57Z

Sorry to pick on plyvel here, the real problem is that LevelDB made an ABI incompatible change in a minor version update (who does that???) google/leveldb#927

It might be possible to add a check to setup.py to add -fno-rtti if the LevelDB version is 1.23 or newer.

wbolster · 2024-09-10T14:18:02Z

Of course, plyvel has some ancient setup.py that doesn't even specify build dependencies or run Cython for you, so don't try to do like every other python package written since 2016 and just pip install . ... you have to run make...

just fyi this seemed sensible, at least in prehistoric times (12 years ago) 🧓🏼 </snark>

see this commit 66028d8

dhdaines · 2024-09-10T14:44:05Z

just fyi this seemed sensible, at least in prehistoric times (12 years ago) 🧓🏼 </snark>

Indeed, it definitely made sense in the stone age of Python packaging when there was no good solution for build dependencies!

liviaerxin · 2024-09-10T14:48:38Z

Hi there. If you need the already built binary plyvel packages for windows, Linux and macOS, please feel free to have a look at #152. It has built and packaged leveldb and snappy within it.

jbothma · 2024-09-27T16:42:46Z

The CFLAGS are not being passed into the build process

with Make also needed to pass the directories via setup.py. There's probably an easier way but this worked when I needed it.

-extra_compile_args = ['-Wall', '-g', '-x', 'c++', '-std=c++11']
+extra_compile_args = ['-Wall', '-g', '-x', 'c++', '-std=c++11', '-fno-rtti']

if platform.system() == 'Darwin':
    extra_compile_args += ['-stdlib=libc++']
@@ -25,6 +25,8 @@ def get_file_contents(filename):
        sources=['plyvel/_plyvel.cpp', 'plyvel/comparator.cpp'],
        libraries=['leveldb'],
        extra_compile_args=extra_compile_args,
+        include_dirs=['/opt/homebrew/Cellar/leveldb/1.23_1/include/'],
+        library_dirs=['/opt/homebrew/Cellar/leveldb/1.23_1/lib/'],
    )
]

wbolster added builds help-wanted labels Apr 26, 2020

rickwierenga mentioned this issue Mar 21, 2021

What does "set the proper environment variables for the compiler and linker" mean? #124

Open

Small25 mentioned this issue Apr 30, 2021

Hello, Small25/mini#13

Open

ksato9700 mentioned this issue Sep 25, 2021

Plyvel not linked correctly to leveldb in MacOS #134

Open

successor1 mentioned this issue Dec 18, 2021

CMakeLists.txt changes for Silicon M1 theQRL/qryptonight#90

Closed

wbolster mentioned this issue Dec 29, 2021

How to enable RTTI without modify leveldb/CMakeList.txt google/leveldb#731

Open

potiuk mentioned this issue Nov 2, 2024

Explain how to use uv with airflow virtualenv and make it works apache/airflow#43604

Merged

maraudersmapx mentioned this issue Dec 11, 2024

build/plyvel/_plyvel.cpp:1293:10: fatal error: leveldb/db.h: No such file or directory prof79/fansly-downloader-ng#88

Open

3 tasks

installation failed on mac osx 10.15 #114

installation failed on mac osx 10.15 #114

Comments

PiPinoccio commented Apr 25, 2020

wbolster commented Apr 26, 2020 • edited Loading

bocheng0000 commented May 8, 2020

bagonyi commented Mar 2, 2021 • edited Loading

sr-gi commented Mar 10, 2021 • edited Loading

sr-gi commented Mar 10, 2021

wbolster commented Mar 11, 2021

sr-gi commented Mar 11, 2021

wbolster commented Mar 11, 2021

sr-gi commented Mar 15, 2021

rickwierenga commented Mar 21, 2021

sr-gi commented Mar 21, 2021

CypherGrue commented Mar 26, 2021 • edited Loading

bagonyi commented Mar 30, 2021

CypherGrue commented Mar 30, 2021

maziyarpanahi commented Apr 9, 2021

liviaerxin commented May 6, 2021 • edited Loading

agatti commented May 6, 2021

ibnachraf commented Sep 13, 2021

YSaxon commented Feb 28, 2022

YSaxon commented Feb 28, 2022

3cola commented Mar 17, 2022

trungtin commented Jun 22, 2022

arlaneenalra commented Sep 27, 2022

Siemingfong commented Jan 24, 2023

nicolasff commented Jun 18, 2023

catap commented Sep 6, 2023

Pandemolde commented Oct 16, 2023

timkpaine commented Jan 6, 2024 • edited Loading

lehrblogger commented Jan 18, 2024

YuraAka commented Jan 25, 2024

markoelez commented Jan 31, 2024

xavi-clovr commented Feb 1, 2024

jonathanykh commented Feb 12, 2024

mnba commented Jun 15, 2024

dhdaines commented Sep 10, 2024 • edited Loading

dhdaines commented Sep 10, 2024 • edited Loading

wbolster commented Sep 10, 2024 • edited Loading

dhdaines commented Sep 10, 2024

liviaerxin commented Sep 10, 2024 • edited Loading

jbothma commented Sep 27, 2024

wbolster commented Apr 26, 2020 •

edited

Loading

bagonyi commented Mar 2, 2021 •

edited

Loading

sr-gi commented Mar 10, 2021 •

edited

Loading

CypherGrue commented Mar 26, 2021 •

edited

Loading

liviaerxin commented May 6, 2021 •

edited

Loading

timkpaine commented Jan 6, 2024 •

edited

Loading

dhdaines commented Sep 10, 2024 •

edited

Loading

dhdaines commented Sep 10, 2024 •

edited

Loading

wbolster commented Sep 10, 2024 •

edited

Loading

liviaerxin commented Sep 10, 2024 •

edited

Loading