Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Overlinking check crashes with TypeError #5123

Closed
marcoesters opened this issue Dec 20, 2023 · 4 comments · Fixed by #5124
Closed

Overlinking check crashes with TypeError #5123

marcoesters opened this issue Dec 20, 2023 · 4 comments · Fixed by #5124
Assignees
@kenodegard
Labels
in-progress issue is actively being worked on locked [bot] locked due to inactivity severity::3 major; broken functionality with a workaround source::anaconda created by members of Anaconda, Inc. type::bug describes erroneous operation, use severity::* to classify the type

Comments

@marcoesters
Copy link

marcoesters commented Dec 20, 2023 •
edited by kenodegard
Loading

What happened?

When trying to build pyinstaller 5.13.2 on linux-s390x, the overlinking check crashes with a TypeError.

To reproduce, use the following recipe: AnacondaRecipes/pyinstaller-feedstock#7

And remove the overlinking_ignore_patterns field.

Traceback

    Traceback (most recent call last):
      File "/opt/conda/lib/python3.9/site-packages/conda/exception_handler.py", line 17, in __call__
        return func(*args, **kwargs)
      File "/opt/conda/lib/python3.9/site-packages/conda/cli/main.py", line 83, in main_subshell
        exit_code = do_call(args, parser)
      File "/opt/conda/lib/python3.9/site-packages/conda/cli/conda_argparse.py", line 170, in do_call
        result = plugin_subcommand.action(getattr(args, "_args", args))
      File "/opt/conda/lib/python3.9/site-packages/conda_build/plugin.py", line 10, in build
        execute(*args, **kwargs)
      File "/opt/conda/lib/python3.9/site-packages/conda_build/cli/main_build.py", line 572, in execute
        outputs = api.build(
      File "/opt/conda/lib/python3.9/site-packages/conda_build/api.py", line 254, in build
        return build_tree(
      File "/opt/conda/lib/python3.9/site-packages/conda_build/build.py", line 3789, in build_tree
        packages_from_this = build(
      File "/opt/conda/lib/python3.9/site-packages/conda_build/build.py", line 2877, in build
        newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
      File "/opt/conda/lib/python3.9/site-packages/conda_build/build.py", line 2004, in bundle_conda
        files = post_process_files(metadata, initial_files)
      File "/opt/conda/lib/python3.9/site-packages/conda_build/build.py", line 1815, in post_process_files
        post_build(m, new_files, build_python=python)
      File "/opt/conda/lib/python3.9/site-packages/conda_build/post.py", line 1819, in post_build
        check_overlinking(m, files, host_prefix)
      File "/opt/conda/lib/python3.9/site-packages/conda_build/post.py", line 1644, in check_overlinking
        return check_overlinking_impl(
      File "/opt/conda/lib/python3.9/site-packages/conda_build/post.py", line 1500, in check_overlinking_impl
        all_needed_dsos, needed_dsos_for_file = _collect_needed_dsos(
      File "/opt/conda/lib/python3.9/site-packages/conda_build/post.py", line 908, in _collect_needed_dsos
        needed = get_linkages_memoized(
      File "/opt/conda/lib/python3.9/site-packages/conda_build/os_utils/liefldd.py", line 1212, in __call__
        value = self.func(*args, **kw)
      File "/opt/conda/lib/python3.9/site-packages/conda_build/os_utils/liefldd.py", line 1241, in get_linkages_memoized
        return get_linkages(
      File "/opt/conda/lib/python3.9/site-packages/conda_build/os_utils/liefldd.py", line 649, in get_linkages
        result_lief = inspect_linkages_lief(
      File "/opt/conda/lib/python3.9/site-packages/conda_build/os_utils/liefldd.py", line 528, in inspect_linkages_lief
        uniqueness_key = get_uniqueness_key(binary)
      File "/opt/conda/lib/python3.9/site-packages/conda_build/os_utils/liefldd.py", line 376, in get_uniqueness_key
        binary = ensure_binary(file)
      File "/opt/conda/lib/python3.9/site-packages/conda_build/os_utils/liefldd.py", line 49, in ensure_binary
        elif not Path(file).exists():
      File "/opt/conda/lib/python3.9/pathlib.py", line 1082, in __new__
        self = cls._from_parts(args, init=False)
      File "/opt/conda/lib/python3.9/pathlib.py", line 707, in _from_parts
        drv, root, parts = self._parse_args(args)
      File "/opt/conda/lib/python3.9/pathlib.py", line 691, in _parse_args
        a = os.fspath(a)
    TypeError: expected str, bytes or os.PathLike object, not NoneType

Conda Details

conda info 
active environment : None                                                                                                
       user config file : /root/.condarc                                                                                      
 populated config files : /opt/conda/.condarc                                                                                 
          conda version : 23.11.0                                                                                             
    conda-build version : 3.28.2                                                                                              
         python version : 3.9.18.final.0                                                                                      
                 solver : libmamba (default)                                                                                  
       virtual packages : __archspec=1=s390x                                                                                  
                          __conda=23.11.0=0                                                                                   
                          __glibc=2.17=0                                                                                      
                          __linux=5.10.124=0                                                                                  
                          __unix=0=0                                                                                          
       base environment : /opt/conda  (writable)                                                                              
      conda av data dir : /opt/conda/etc/conda                                                                                
  conda av metadata url : None                                                                                                
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-s390x                                                     
                          https://repo.anaconda.com/pkgs/main/noarch                                                          
                          https://repo.anaconda.com/pkgs/r/linux-s390x                                                        
                          https://repo.anaconda.com/pkgs/r/noarch                                                             
          package cache : /opt/conda/pkgs                                                                                     
                          /root/.conda/pkgs                                                                                   
       envs directories : /opt/conda/envs                                                                                     
                          /root/.conda/envs                                                                                   
               platform : linux-s390x                                                                                         
             user-agent : conda/23.11.0 requests/2.31.0 CPython/3.9.18 Linux/5.10.124-linuxkit centos/7.7.1908 glibc/2.17 solver/libmamba conda-libmamba-solver/23.12.0 libmambapy/1.5.3                                                                    
                UID:GID : 0:0                                                                                                 
             netrc file : None                                                                                                
           offline mode : False
conda config 
==> /opt/conda/.condarc <==                                                                                                   
add_pip_as_python_dependency: False                                                                                           
show_channel_urls: True
conda list 
# packages in environment at /opt/conda:                                                                                      
#                                                                                                                             
# Name                    Version                   Build  Channel                                                            
_libgcc_mutex             0.1                        main    defaults                                                         
_openmp_mutex             5.1                       1_gnu    defaults                                                         
archspec                  0.2.1              pyhd3eb1b0_0    defaults                                                         
attrs                     23.1.0           py39ha847dfd_0    defaults                                                         
beautifulsoup4            4.12.2           py39ha847dfd_0    defaults                                                         
boltons                   23.0.0           py39ha847dfd_0    defaults                                                         
brotli-python             1.0.9            py39hb314cc7_7    defaults                                                         
bzip2                     1.0.8                h2a837d6_0    defaults                                                         
c-ares                    1.19.1               hc33a586_0    defaults                                                         
ca-certificates           2023.12.12           ha847dfd_0    defaults                                                         
certifi                   2023.11.17       py39ha847dfd_0    defaults
cffi                      1.16.0           py39hc33a586_0    defaults
chardet                   4.0.0           py39ha847dfd_1003    defaults
charset-normalizer        2.0.4              pyhd3eb1b0_0    defaults
click                     8.1.7            py39ha847dfd_0    defaults
conda                     23.11.0          py39ha847dfd_0    defaults
conda-build               3.28.2           py39ha847dfd_0    defaults
conda-content-trust       0.1.3            py39ha847dfd_0    defaults
conda-index               0.3.0            py39ha847dfd_0    defaults
conda-libmamba-solver     23.12.0            pyhd3eb1b0_1    defaults
conda-package-handling    2.2.0            py39ha847dfd_0    defaults
conda-package-streaming   0.9.0            py39ha847dfd_0    defaults
cryptography              41.0.7           py39h0f35eb8_0    defaults
distro                    1.8.0            py39ha847dfd_0    defaults
filelock                  3.13.1           py39ha847dfd_0    defaults
fmt                       9.1.0                h36a787c_0    defaults
icu                       73.1                 hb314cc7_0    defaults
idna                      3.3                pyhd3eb1b0_0    defaults
jinja2                    3.1.2            py39ha847dfd_0    defaults
jsonpatch                 1.32               pyhd3eb1b0_0    defaults
jsonpointer               2.1                pyhd3eb1b0_0    defaults
jsonschema                4.19.2           py39ha847dfd_0    defaults
jsonschema-specifications 2023.7.1         py39ha847dfd_0    defaults
krb5                      1.20.1               hb8112af_1    defaults
ld_impl_linux-s390x       2.38                 h2beb32a_1    defaults
libarchive                3.6.2                h9384daa_2    defaults
libcurl                   8.4.0                hf098187_1    defaults
libedit                   3.1.20230828         hc33a586_0    defaults
libev                     4.33                 h2a837d6_1    defaults
libffi                    3.4.2                hb314cc7_6    defaults
libgcc-ng                 11.2.0               h1234567_1    defaults
libgomp                   11.2.0               h1234567_1    defaults
liblief                   0.12.3               hb314cc7_0    defaults
libmamba                  1.5.3                h2a17aea_0    defaults
libmambapy                1.5.3            py39ha78da6f_0    defaults
libnghttp2                1.57.0               hadef098_0    defaults
libsolv                   0.7.24               h92bb072_0    defaults
libssh2                   1.10.0               h2f79a9f_2    defaults
libstdcxx-ng              11.2.0               h1234567_1    defaults
libxml2                   2.10.4               h2b138d1_1    defaults
lz4-c                     1.9.4                hb314cc7_0    defaults
markupsafe                2.1.1            py39h2a837d6_0    defaults
menuinst                  2.0.1            py39ha847dfd_1    defaults
more-itertools            10.1.0           py39ha847dfd_0    defaults
ncurses                   6.4                  hb314cc7_0    defaults
openssl                   3.0.12               h2a837d6_0    defaults
packaging                 23.1             py39ha847dfd_0    defaults
patch                     2.7.6             h2a837d6_1001    defaults
patchelf                  0.17.2               hb314cc7_0    defaults
pcre2                     10.42                h51f2fbf_0    defaults
pip                       23.0.1           py39ha847dfd_0    defaults
pkginfo                   1.9.6            py39ha847dfd_0    defaults
platformdirs              3.10.0           py39ha847dfd_0    defaults
pluggy                    1.0.0            py39ha847dfd_1    defaults
psutil                    5.9.0            py39hc33a586_0    defaults
py-lief                   0.12.3           py39hb314cc7_0    defaults
pybind11-abi              4                    hd3eb1b0_1    defaults
pycosat                   0.6.6            py39hc33a586_0    defaults
pycparser                 2.21               pyhd3eb1b0_0    defaults
pyopenssl                 23.2.0           py39ha847dfd_0    defaults
pysocks                   1.7.1            py39ha847dfd_0    defaults
python                    3.9.18               hb93da09_0    defaults
python-libarchive-c       2.9                pyhd3eb1b0_1    defaults
pytz                      2023.3.post1     py39ha847dfd_0    defaults
pyyaml                    6.0.1            py39hc33a586_0    defaults
readline                  8.2                  hc33a586_0    defaults
referencing               0.30.2           py39ha847dfd_0    defaults
reproc                    14.2.4               hd8db5ea_1    defaults
reproc-cpp                14.2.4               hd8db5ea_1    defaults
requests                  2.31.0           py39ha847dfd_0    defaults
rpds-py                   0.10.6           py39h01014ec_0    defaults
ruamel.yaml               0.17.21          py39hc33a586_0    defaults
ruamel.yaml.clib          0.2.6            py39hc33a586_1    defaults
setuptools                65.6.3           py39ha847dfd_0    defaults
six                       1.16.0             pyhd3eb1b0_1    defaults
soupsieve                 2.5              py39ha847dfd_0    defaults
sqlite                    3.41.2               hc33a586_0    defaults
tk                        8.6.12               h2795051_0    defaults
tomli                     2.0.1            py39ha847dfd_0    defaults
tqdm                      4.65.0           py39heb6ab9f_0    defaults
tzdata                    2023c                h04d1e81_0    defaults
urllib3                   1.26.18          py39ha847dfd_0    defaults
wheel                     0.38.4           py39ha847dfd_0    defaults
xz                        5.4.5                hc33a586_0    defaults
yaml                      0.2.5                h2a837d6_0    defaults
yaml-cpp                  0.8.0                hb314cc7_0    defaults
zlib                      1.2.13               hc33a586_0    defaults
zstandard                 0.19.0           py39hc33a586_0    defaults
zstd                      1.5.5                h99a12f6_0    defaults
@marcoesters marcoesters added the type::bug describes erroneous operation, use severity::* to classify the type label Dec 20, 2023
@github-project-automation github-project-automation bot moved this to 🆕 New in 🧭 Planning Dec 20, 2023
@marcoesters marcoesters mentioned this issue Dec 20, 2023
Merged
@kenodegard kenodegard added source::anaconda created by members of Anaconda, Inc. severity::1 blocker; broken functionality with no workaround in-progress issue is actively being worked on labels Dec 20, 2023
@kenodegard kenodegard mentioned this issue Dec 20, 2023
Merged
3 tasks
@kenodegard kenodegard moved this from 🆕 New to 🏗️ In Progress in 🧭 Planning Dec 20, 2023
@kenodegard kenodegard self-assigned this Dec 20, 2023
@kenodegard kenodegard added severity::3 major; broken functionality with a workaround and removed severity::1 blocker; broken functionality with no workaround labels Dec 20, 2023
@kenodegard
Copy link
Contributor

Downgrade to severity::3 major; broken functionality with a workaround since the overlinking checks can be skipped: https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#ignoring-files-in-overlinking-overdepending-checks

@mbargull
Copy link
Member

I've encountered this too in https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=849129&view=logs&j=a70f640f-cc53-5cd3-6cdc-236a1aa90802&t=6119ccbe-9301-594f-7c27-f792b80a7fcc&l=18498 .

Do we have a rough plan regarding a new patch release?
(Asking because of the holiday downtime which makes planning harder to predict..)

mbargull added a commit to mbargull-feedstocks/perl-feedstock that referenced this issue Dec 23, 2023
@mbargull
Use conda-build=3.27 temporarily
1a8e80e 
Due to failing Windows build that hit
conda/conda-build#5123 .

Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
mbargull added a commit to mbargull-feedstocks/perl-feedstock that referenced this issue Dec 23, 2023
@mbargull
Use conda-build=3.27 temporarily
517e6d6 
Due to failing Windows build that hit
conda/conda-build#5123 .

Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
@kenodegard
Copy link
Contributor

@mbargull I'll start a release on Tuesday

@kenodegard kenodegard linked a pull request Dec 26, 2023 that will close this issue
Fix conda_build.os_utils.liefldd.ensure_binary to handle None #5124
Merged
3 tasks
@kenodegard
Copy link
Contributor

#5124 has been merged

@github-project-automation github-project-automation bot moved this from 🏗️ In Progress to 🏁 Done in 🧭 Planning Dec 26, 2023
@github-actions github-actions bot added the locked [bot] locked due to inactivity label Aug 20, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@kenodegard kenodegard

Labels
in-progress issue is actively being worked on locked [bot] locked due to inactivity severity::3 major; broken functionality with a workaround source::anaconda created by members of Anaconda, Inc. type::bug describes erroneous operation, use severity::* to classify the type
Projects
🧭 Planning
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix conda_build.os_utils.liefldd.ensure_binary to handle None kenodegard/conda-build
3 participants
@kenodegard @mbargull @marcoesters