Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-fvisibility-inlines-hidden breaks libprotobuf (and potentially other software) #73

Open
1 task done
pitrou opened this issue May 11, 2022 · 14 comments
Open
1 task done
Labels
bug Something isn't working

Comments

@pitrou
Copy link
Member

pitrou commented May 11, 2022

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

The -fvisibility-inlines-hidden compiler option is described thusly in the GCC wiki:

This causes all inlined class member functions to have hidden visibility, causing significant export symbol table size & binary size reductions

This looks theoretically harmless, except that whether class member are defined inline may depend on exact compilation options.

One typical example is when a function declaration depends on whether NDEBUG is enabled. This is exactly what libprotobuf 3.20.1 does (but not 3.19.4):

#if defined(NDEBUG) || defined(_MSC_VER)
  ~InternalMetadata() {
    if (HasMessageOwnedArenaTag()) {
      delete reinterpret_cast<Arena*>(ptr_ - kMessageOwnedArenaTagMask);
    }
  }
#else
  ~InternalMetadata();
#endif

What happens is then:

  • conda-forge compiles libprotobuf 3.20.1 with NDEBUG defined (release mode), so ~InternalMetadata, being inlined, isn't exported by libprotobuf.so
  • third-party software, when compiling and linking in debug mode against libprotobuf (so with NDEBUG not defined), expects to find the ~InternalMetadata in libprotobuf.so, and fails

Installed packages

# packages in environment at /home/antoine/miniconda3/envs/pyarrow:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
_sysroot_linux-64_curr_repodata_hack 3                   h5bd9786_13    conda-forge
abseil-cpp                20211102.0           h27087fc_1    conda-forge
aiobotocore               1.4.2                    pypi_0    pypi
aiohttp                   3.7.4.post0              pypi_0    pypi
aioitertools              0.8.0                    pypi_0    pypi
alabaster                 0.7.12                   pypi_0    pypi
archery                   0.1.0                     dev_0    <develop>
asttokens                 2.0.5              pyhd8ed1ab_0    conda-forge
async-timeout             3.0.1                    pypi_0    pypi
attrs                     21.2.0                   pypi_0    pypi
autopep8                  1.5.7                    pypi_0    pypi
aws-c-auth                0.6.11              h2f9c894_10    conda-forge
aws-c-cal                 0.5.17               h6b4091c_0    conda-forge
aws-c-common              0.6.20               h166bdaf_0    conda-forge
aws-c-compression         0.2.14              h8333888_10    conda-forge
aws-c-event-stream        0.2.7               h3c68b06_43    conda-forge
aws-c-http                0.6.14               h6db1ea3_1    conda-forge
aws-c-io                  0.10.22              h991c0f4_0    conda-forge
aws-c-mqtt                0.7.10              h9f869e8_15    conda-forge
aws-c-s3                  0.1.38               h1c0cbb8_4    conda-forge
aws-c-sdkutils            0.1.2                h8333888_1    conda-forge
aws-checksums             0.1.12               h8333888_9    conda-forge
aws-crt-cpp               0.17.28              h070cc90_2    conda-forge
aws-sdk-cpp               1.9.253              h0519c9a_0    conda-forge
babel                     2.9.1                    pypi_0    pypi
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.1                pyhd3eb1b0_0    defaults
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.10.0                   pypi_0    pypi
benchmark                 1.6.1                h9c3ff4c_0    conda-forge
binutils                  2.36.1               hdd6e379_2    conda-forge
binutils_impl_linux-64    2.36.1               h193b22a_2    conda-forge
binutils_linux-64         2.36                 hf3e587d_9    conda-forge
black                     22.3.0             pyhd8ed1ab_0    conda-forge
blinker                   1.4                      pypi_0    pypi
boost-cpp                 1.71.0               h7f8727e_1    defaults
botocore                  1.20.106                 pypi_0    pypi
breathe                   4.31.0                   pypi_0    pypi
brotli                    1.0.9                h166bdaf_7    conda-forge
brotli-bin                1.0.9                h166bdaf_7    conda-forge
brotlipy                  0.7.0                    pypi_0    pypi
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
c-compiler                1.4.2                h166bdaf_0    conda-forge
ca-certificates           2022.4.26            h06a4308_0    defaults
certifi                   2021.10.8        py39hf3d152e_2    conda-forge
cffi                      1.14.6                   pypi_0    pypi
chardet                   4.0.0                    pypi_0    pypi
charset-normalizer        2.0.7                    pypi_0    pypi
clang                     13.0.1               ha770c72_0    conda-forge
clang-13                  13.0.1          default_hc23dcda_0    conda-forge
clangxx                   13.0.1          default_ha53f305_0    conda-forge
click                     8.0.3                    pypi_0    pypi
cmake                     3.23.1               h5432695_0    conda-forge
cmake-format              0.6.13                   pypi_0    pypi
cmakelang                 0.6.13                   pypi_0    pypi
compilers                 1.4.2                ha770c72_0    conda-forge
contextlib2               21.6.0                   pypi_0    pypi
crc32c                    2.2                      pypi_0    pypi
cryptography              35.0.0                   pypi_0    pypi
cxx-compiler              1.4.2                h924138e_0    conda-forge
cython                    0.29.28          py39h5a03fae_2    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1                    pypi_0    pypi
deprecated                1.2.13                   pypi_0    pypi
docutils                  0.16                     pypi_0    pypi
doxygen                   1.9.2                hb166930_0    conda-forge
executing                 0.8.3              pyhd8ed1ab_0    conda-forge
expat                     2.4.8                h27087fc_0    conda-forge
flake8                    4.0.1                    pypi_0    pypi
flask                     2.0.1                    pypi_0    pypi
flatbuffers               2.0.6                h27087fc_0    conda-forge
fortran-compiler          1.4.2                h2a4ca65_0    conda-forge
fsspec                    2021.10.0                pypi_0    pypi
gcc                       10.3.0               he2824d0_9    conda-forge
gcc_impl_linux-64         10.3.0              hf2f2afa_16    conda-forge
gcc_linux-64              10.3.0               hc39de41_9    conda-forge
gflags                    2.2.2             he1b5a44_1004    conda-forge
gfortran                  10.3.0               h18518b4_9    conda-forge
gfortran_impl_linux-64    10.3.0              h73f4979_16    conda-forge
gfortran_linux-64         10.3.0               hb09a455_9    conda-forge
gitdb                     4.0.7                    pypi_0    pypi
github3-py                2.0.0                    pypi_0    pypi
gitpython                 3.1.24                   pypi_0    pypi
glog                      0.6.0                h6f12383_0    conda-forge
gmock                     1.11.0               h924138e_0    conda-forge
google-cloud-cpp          1.40.1               hf2015ea_0    conda-forge
googleapis-common-protos  1.53.0                   pypi_0    pypi
googleapis-cpp            0.10.0              haea4d6a_24    conda-forge
googleapis-storage-testbench 0.7.0                    pypi_0    pypi
grpc-cpp                  1.45.2               he70e3f0_3    conda-forge
grpcio                    1.41.0                   pypi_0    pypi
gtest                     1.11.0               h924138e_0    conda-forge
gunicorn                  20.1.0                   pypi_0    pypi
gxx                       10.3.0               he2824d0_9    conda-forge
gxx_impl_linux-64         10.3.0              hf2f2afa_16    conda-forge
gxx_linux-64              10.3.0               h2593f52_9    conda-forge
httpbin                   0.7.0                    pypi_0    pypi
hypothesis                6.23.2                   pypi_0    pypi
icu                       58.2              hf484d3e_1000    conda-forge
idna                      3.3                      pypi_0    pypi
imagesize                 1.2.0                    pypi_0    pypi
importlib-metadata        4.8.1                    pypi_0    pypi
iniconfig                 1.1.1                    pypi_0    pypi
ipython                   8.3.0            py39hf3d152e_0    conda-forge
itsdangerous              2.0.1                    pypi_0    pypi
jedi                      0.18.1           py39hf3d152e_1    conda-forge
jeepney                   0.7.1                    pypi_0    pypi
jinja2                    3.0.2                    pypi_0    pypi
jira                      3.0.1                    pypi_0    pypi
jmespath                  0.10.0                   pypi_0    pypi
jwcrypto                  1.0                      pypi_0    pypi
kernel-headers_linux-64   3.10.0              h4a8ded7_13    conda-forge
keyring                   23.2.1                   pypi_0    pypi
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.19.3               h3790be6_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libblas                   3.9.0           14_linux64_openblas    conda-forge
libboost                  1.71.0               haf77d95_1    defaults
libbrotlicommon           1.0.9                h166bdaf_7    conda-forge
libbrotlidec              1.0.9                h166bdaf_7    conda-forge
libbrotlienc              1.0.9                h166bdaf_7    conda-forge
libcblas                  3.9.0           14_linux64_openblas    conda-forge
libclang-cpp13            13.0.1          default_hc23dcda_0    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcurl                   7.83.0               h7bff187_0    conda-forge
libedit                   3.1.20210910         h7f8727e_0    defaults
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               h9b69904_4    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-devel_linux-64     10.3.0              he6cfe16_16    conda-forge
libgcc-ng                 11.2.0              h1d223b6_16    conda-forge
libgfortran-ng            11.2.0              h69a702a_16    conda-forge
libgfortran5              11.2.0              h5c6108e_16    conda-forge
libgomp                   11.2.0              h1d223b6_16    conda-forge
libgoogle-cloud           1.40.1               hf2015ea_0    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0           14_linux64_openblas    conda-forge
libllvm11                 11.1.0               hf817b99_3    conda-forge
libllvm13                 13.0.1               hf817b99_2    conda-forge
libnghttp2                1.47.0               h727a467_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libopenblas               0.3.20          pthreads_h78a6416_0    conda-forge
libprotobuf               3.20.1               h6239696_0    conda-forge
libsanitizer              10.3.0              h26c7422_16    conda-forge
libssh2                   1.10.0               ha56f1ee_2    conda-forge
libstdcxx-devel_linux-64  10.3.0              he6cfe16_16    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_16    conda-forge
libthrift                 0.16.0               h519c5ea_1    conda-forge
libutf8proc               2.7.0                h7f98852_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libuv                     1.43.0               h7f98852_0    conda-forge
libzlib                   1.2.11            h166bdaf_1014    conda-forge
llvmlite                  0.38.0           py39h7d9a04d_1    conda-forge
lz4                       4.0.0            py39h029007f_1    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
markupsafe                2.0.1                    pypi_0    pypi
matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge
mccabe                    0.6.1                    pypi_0    pypi
multidict                 5.2.0                    pypi_0    pypi
mypy_extensions           0.4.3            py39hf3d152e_5    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
nlohmann_json             3.10.5               h9c3ff4c_0    conda-forge
numba                     0.55.1           py39h56b8d98_0    conda-forge
numpy                     1.21.6           py39h18676bf_0    conda-forge
numpydoc                  1.1.0                    pypi_0    pypi
oauthlib                  3.1.1                    pypi_0    pypi
openssl                   1.1.1o               h166bdaf_0    conda-forge
orc                       1.7.4                h6c59b99_0    conda-forge
packaging                 21.0                     pypi_0    pypi
pandas                    1.4.2            py39h1832856_1    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pathspec                  0.9.0              pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pickleshare               0.7.5           py39hde42818_1002    conda-forge
pip                       22.0.4             pyhd8ed1ab_0    conda-forge
platformdirs              2.5.1              pyhd8ed1ab_0    conda-forge
pluggy                    1.0.0                    pypi_0    pypi
prompt-toolkit            3.0.29             pyha770c72_0    conda-forge
prompt_toolkit            3.0.29               hd8ed1ab_0    conda-forge
protobuf                  3.18.0                   pypi_0    pypi
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py                        1.10.0                   pypi_0    pypi
pyarrow                   8.0.0.dev533+g7a69bbf18           dev_0    <develop>
pycodestyle               2.8.0                    pypi_0    pypi
pycparser                 2.20                     pypi_0    pypi
pydata-sphinx-theme       0.7.2                    pypi_0    pypi
pyflakes                  2.4.0                    pypi_0    pypi
pygit2                    1.7.0                    pypi_0    pypi
pygithub                  1.55                     pypi_0    pypi
pygments                  2.12.0             pyhd8ed1ab_0    conda-forge
pyjwt                     2.2.0                    pypi_0    pypi
pynacl                    1.4.0                    pypi_0    pypi
pyparsing                 2.4.7                    pypi_0    pypi
pytest                    6.2.5                    pypi_0    pypi
pytest-lazy-fixture       0.6.3                    pypi_0    pypi
pytest-repeat             0.9.1                    pypi_0    pypi
python                    3.9.12          h9a8a25e_1_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-dotenv             0.19.1                   pypi_0    pypi
python_abi                3.9                      2_cp39    conda-forge
pytz                      2022.1             pyhd8ed1ab_0    conda-forge
rapidjson                 1.1.0             he1b5a44_1002    conda-forge
raven                     6.10.0                   pypi_0    pypi
re2                       2022.04.01           h27087fc_0    conda-forge
readline                  8.1.2                h7f8727e_1    defaults
requests                  2.26.0                   pypi_0    pypi
requests-oauthlib         1.3.0                    pypi_0    pypi
requests-toolbelt         0.9.1                    pypi_0    pypi
responses                 0.16.0                   pypi_0    pypi
rhash                     1.4.1                h7f98852_0    conda-forge
ruamel-yaml               0.17.16                  pypi_0    pypi
ruamel-yaml-clib          0.2.6                    pypi_0    pypi
s2n                       1.3.11               h8d01263_0    conda-forge
s3fs                      2021.10.0                pypi_0    pypi
scalpl                    0.4.2                    pypi_0    pypi
schema                    0.7.4                    pypi_0    pypi
scipy                     1.8.0            py39hee8e79c_1    conda-forge
secretstorage             3.3.1                    pypi_0    pypi
semver                    2.13.0                   pypi_0    pypi
setuptools                62.1.0           py39hf3d152e_0    conda-forge
setuptools-scm            6.3.2                    pypi_0    pypi
six                       1.16.0             pyh6c4a22f_0    conda-forge
smmap                     4.0.0                    pypi_0    pypi
snappy                    1.1.9                hbd366e4_0    conda-forge
snowballstemmer           2.1.0                    pypi_0    pypi
sortedcontainers          2.4.0                    pypi_0    pypi
soupsieve                 2.2.1                    pypi_0    pypi
sphinx                    4.3.2                    pypi_0    pypi
sphinx-tabs               3.2.0                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.2                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.2                    pypi_0    pypi
sphinxcontrib-htmlhelp    2.0.0                    pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-qthelp      1.0.3                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.5                    pypi_0    pypi
sqlite                    3.38.5               h4ff8645_0    conda-forge
stack_data                0.2.0              pyhd8ed1ab_0    conda-forge
sysroot_linux-64          2.17                h4a8ded7_13    conda-forge
thrift-compiler           0.16.0               h519c5ea_1    conda-forge
thrift-cpp                0.16.0               h3e2b116_1    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
toml                      0.10.2                   pypi_0    pypi
tomli                     1.2.1                    pypi_0    pypi
traitlets                 5.1.1              pyhd8ed1ab_0    conda-forge
typed-ast                 1.5.3            py39hb9d737c_0    conda-forge
typing-extensions         3.10.0.2                 pypi_0    pypi
typing_extensions         4.2.0              pyha770c72_1    conda-forge
tzdata                    2022a                h191b570_0    conda-forge
uritemplate               4.1.1                    pypi_0    pypi
urllib3                   1.26.7                   pypi_0    pypi
valgrind                  3.18.1               hf817b99_0    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
werkzeug                  2.0.2                    pypi_0    pypi
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
wrapt                     1.13.2                   pypi_0    pypi
xz                        5.2.5                h516909a_1    conda-forge
yarl                      1.7.0                    pypi_0    pypi
zipp                      3.6.0                    pypi_0    pypi
zlib                      1.2.11            h166bdaf_1014    conda-forge
zstd                      1.5.2                ha95c52a_0    conda-forge

Environment info

active environment : pyarrow
    active env location : /home/antoine/miniconda3/envs/pyarrow
            shell level : 1
       user config file : /home/antoine/.condarc
 populated config files : /home/antoine/.condarc
          conda version : 4.12.0
    conda-build version : 3.21.8
         python version : 3.7.12.final.0
       virtual packages : __cuda=11.4=0
                          __linux=5.4.0=0
                          __glibc=2.31=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/antoine/miniconda3  (writable)
      conda av data dir : /home/antoine/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/antoine/miniconda3/pkgs
                          /home/antoine/.conda/pkgs
       envs directories : /home/antoine/miniconda3/envs
                          /home/antoine/.conda/envs
               platform : linux-64
             user-agent : conda/4.12.0 requests/2.27.1 CPython/3.7.12 Linux/5.4.0-109-generic ubuntu/20.04.4 glibc/2.31
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
@pitrou pitrou added the bug Something isn't working label May 11, 2022
@pitrou
Copy link
Member Author

pitrou commented May 11, 2022

Also cc @xhochy @h-vetinari @jakirkham as libprotobuf maintainers (since that package is affected).

@xhochy
Copy link
Member

xhochy commented May 11, 2022

cc @conda-forge/libprotobuf

@pitrou
Copy link
Member Author

pitrou commented May 11, 2022

I also reported the issue to the protobuf issue tracker: protocolbuffers/protobuf#9947

@xhochy
Copy link
Member

xhochy commented May 11, 2022

@pitrou Can you post the compiler error you get? I fail to build a minimal reproducible example and I also struggle to understand what could go wrong here.

@pitrou
Copy link
Member Author

pitrou commented May 11, 2022

There is no compiler error. The error happens at runtime:

$ ./build-test/debug/arrow-flight-test 
./build-test/debug/arrow-flight-test: symbol lookup error: /home/antoine/miniconda3/envs/pyarrow/lib/libarrow_flight.so.900: undefined symbol: _ZN6google8protobuf8internal16InternalMetadataD1Ev
$ c++filt _ZN6google8protobuf8internal16InternalMetadataD1Ev
google::protobuf::internal::InternalMetadata::~InternalMetadata()

@xhochy
Copy link
Member

xhochy commented May 11, 2022

This should have been caught by the linker earlier. Also this would mean that you could not link against any destructor in a lib. The above code changes behaviour depending on the definitions but it doesn't change inline-vs-not-inline.

@pitrou
Copy link
Member Author

pitrou commented May 11, 2022

The code does change inline-vs-not-inline (a member function which is defined inside the class body is implicitly inline).

@xhochy
Copy link
Member

xhochy commented May 11, 2022

Ah, I implicitly read ~InternalMetadata() {}

@xhochy
Copy link
Member

xhochy commented May 11, 2022

Keeping the discussion for now in the upstream Arrow issue until we fully understand what is going wrong here.

@xhochy
Copy link
Member

xhochy commented May 12, 2022

Ok, here is a reproducer: https://gist.github.com/pitrou/1c892d1c42bd6ece9b68a9eb015b76c8

@pitrou
Copy link
Member Author

pitrou commented May 17, 2022

protocolbuffers/protobuf#9947 was closed as won't fix so this will have to be fixed on the conda-forge side.

@jakirkham
Copy link
Member

Some packages need to tweak the compiler flags (for example). Would suggest just making this change in the feedstock for now

benmwebb added a commit to salilab/npctransport that referenced this issue Jun 3, 2022
Builds are failing with protobuf 3.20.1 at runtime with
ImportError: libimp_npctransport.so.0: undefined symbol:
_ZN6google8protobuf8internal16InternalMetadataD1Ev

This appears to be because protobuf was built in release
mode (with NDEBUG defined) but we are (perhaps) building
in debug mode (without NDEBUG). Relates
conda-forge/ctng-compiler-activation-feedstock#73.
@pitrou
Copy link
Member Author

pitrou commented Jun 28, 2022

Not wanting to put any pressure, but it would be nice to get this fixed at some point :-)

@pitrou
Copy link
Member Author

pitrou commented Jul 20, 2022

Finally, a fix was accepted for protocolbuffers/protobuf#9947 and should be in the upcoming protobuf 21.3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants