Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem installing pentads with conda #11

Open
rlisenhart opened this issue Sep 22, 2022 · 24 comments
Open

Problem installing pentads with conda #11

rlisenhart opened this issue Sep 22, 2022 · 24 comments

Comments

@rlisenhart
Copy link

Hello, I am having trouble installing pentads. I followed the install instructions, but when building the conda environment I get a series of errors. The first one appears to be:

Failed to build cooltools cytoolz pypairix

Which then cause a downstream series of errors. I'm wondering if you have any advice to solve this issue? I've included the entire error below.
Best,
Randi

Collecting package metadata (repodata.json): done
Solving environment: done
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: - Ran pip subprocess with arguments:
['/Users/isenhart/opt/anaconda3/envs/pentads/bin/python', '-m', 'pip', 'install', '-U', '-r', '/Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt']
Pip subprocess output:
Collecting cooler>=0.8.5
Using cached cooler-0.8.11-py2.py3-none-any.whl (100 kB)
Collecting cooltools>=0.5.0
Using cached cooltools-0.5.1.tar.gz (8.0 MB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting multiprocess
Using cached multiprocess-0.70.13-py37-none-any.whl (115 kB)
Requirement already satisfied: pandas in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from cooler>=0.8.5->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 1)) (1.3.5)
Collecting h5py>=2.5
Using cached h5py-3.7.0-cp37-cp37m-macosx_10_9_x86_64.whl (3.1 MB)
Collecting click>=7
Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting cytoolz<0.11
Using cached cytoolz-0.10.1.tar.gz (475 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: scipy>=0.16 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from cooler>=0.8.5->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 1)) (1.7.3)
Collecting pyfaidx
Using cached pyfaidx-0.7.1-py3-none-any.whl
Collecting asciitree
Using cached asciitree-0.3.3-py3-none-any.whl
Collecting pyyaml
Using cached PyYAML-6.0-cp37-cp37m-macosx_10_9_x86_64.whl (189 kB)
Collecting simplejson
Using cached simplejson-3.17.6-cp37-cp37m-macosx_10_9_x86_64.whl (74 kB)
Collecting pypairix
Using cached pypairix-0.3.7.tar.gz (65 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: six in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from cooler>=0.8.5->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 1)) (1.16.0)
Requirement already satisfied: numpy>=1.9 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from cooler>=0.8.5->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 1)) (1.21.6)
Collecting bioframe>=0.3.3
Using cached bioframe-0.3.3-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: cython in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (0.29.32)
Collecting joblib
Using cached joblib-1.2.0-py3-none-any.whl (297 kB)
Requirement already satisfied: matplotlib in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (3.5.3)
Collecting numba
Using cached numba-0.56.2-cp37-cp37m-macosx_10_14_x86_64.whl (2.4 MB)
Collecting scikit-learn
Using cached scikit_learn-1.0.2-cp37-cp37m-macosx_10_13_x86_64.whl (7.8 MB)
Collecting scikit-image
Using cached scikit_image-0.19.3-cp37-cp37m-macosx_10_13_x86_64.whl (13.2 MB)
Collecting requests
Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting importlib-metadata
Using cached importlib_metadata-4.12.0-py3-none-any.whl (21 kB)
Collecting toolz>=0.8.0
Using cached toolz-0.12.0-py3-none-any.whl (55 kB)
Requirement already satisfied: python-dateutil>=2.7.3 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from pandas->cooler>=0.8.5->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from pandas->cooler>=0.8.5->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 1)) (2022.2.1)
Requirement already satisfied: packaging>=20.0 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from matplotlib->cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (21.3)
Requirement already satisfied: cycler>=0.10 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from matplotlib->cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from matplotlib->cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (4.37.3)
Requirement already satisfied: pyparsing>=2.2.1 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from matplotlib->cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (3.0.9)
Requirement already satisfied: kiwisolver>=1.0.1 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from matplotlib->cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (1.4.4)
Requirement already satisfied: pillow>=6.2.0 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from matplotlib->cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (9.2.0)
Collecting dill>=0.3.5.1
Using cached dill-0.3.5.1-py2.py3-none-any.whl (95 kB)
Collecting llvmlite<0.40,>=0.39.0dev0
Using cached llvmlite-0.39.1-cp37-cp37m-macosx_10_9_x86_64.whl (25.5 MB)
Requirement already satisfied: setuptools<60 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from numba->cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (59.8.0)
Collecting networkx>=2.2
Using cached networkx-2.6.3-py3-none-any.whl (1.9 MB)
Collecting imageio>=2.4.1
Using cached imageio-2.22.0-py3-none-any.whl (3.4 MB)
Collecting tifffile>=2019.7.26
Using cached tifffile-2021.11.2-py3-none-any.whl (178 kB)
Collecting PyWavelets>=1.1.1
Using cached PyWavelets-1.3.0-cp37-cp37m-macosx_10_13_x86_64.whl (4.3 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Requirement already satisfied: typing-extensions in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from kiwisolver>=1.0.1->matplotlib->cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (4.3.0)
Collecting zipp>=0.5
Using cached zipp-3.8.1-py3-none-any.whl (5.6 kB)
Requirement already satisfied: certifi>=2017.4.17 in /Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages (from requests->bioframe>=0.3.3->cooltools>=0.5.0->-r /Users/isenhart/pentads/condaenv.c7jpgos3.requirements.txt (line 2)) (2022.9.14)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.4-py3-none-any.whl (61 kB)
Building wheels for collected packages: cooltools, cytoolz, pypairix
Building wheel for cooltools (setup.py): started
Building wheel for cooltools (setup.py): finished with status 'error'
Running setup.py clean for cooltools
Building wheel for cytoolz (setup.py): started
Building wheel for cytoolz (setup.py): finished with status 'error'
Running setup.py clean for cytoolz
Building wheel for pypairix (setup.py): started
Building wheel for pypairix (setup.py): finished with status 'error'
Running setup.py clean for pypairix
Failed to build cooltools cytoolz pypairix
Installing collected packages: pypairix, asciitree, zipp, urllib3, toolz, tifffile, threadpoolctl, simplejson, pyyaml, PyWavelets, pyfaidx, networkx, llvmlite, joblib, imageio, idna, h5py, dill, charset-normalizer, scikit-learn, scikit-image, requests, multiprocess, importlib-metadata, cytoolz, numba, click, bioframe, cooler, cooltools
Running setup.py install for pypairix: started
Running setup.py install for pypairix: finished with status 'error'

Pip subprocess error:
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [59 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.9-x86_64-3.7
creating build/lib.macosx-10.9-x86_64-3.7/cooltools
copying cooltools/init.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools
copying cooltools/main.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools
creating build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/saddle.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/eigs_cis.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/util.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/eigs_trans.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/dots.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/init.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/insulation.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/pileup.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/expected_cis.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/sample.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/logbin_expected.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/expected_trans.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
copying cooltools/cli/genome.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/cli
creating build/lib.macosx-10.9-x86_64-3.7/cooltools/sandbox
copying cooltools/sandbox/init.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/sandbox
copying cooltools/sandbox/expected_smoothing.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/sandbox
copying cooltools/sandbox/cool2cworld.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/sandbox
copying cooltools/sandbox/pairs_scaling_functions.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/sandbox
copying cooltools/sandbox/balance.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/sandbox
copying cooltools/sandbox/contrast.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/sandbox
creating build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
copying cooltools/lib/peaks.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
copying cooltools/lib/checks.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
copying cooltools/lib/numutils.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
copying cooltools/lib/plotting.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
copying cooltools/lib/_query.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
copying cooltools/lib/io.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
copying cooltools/lib/init.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
copying cooltools/lib/schemas.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
copying cooltools/lib/common.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
copying cooltools/lib/runlength.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/lib
creating build/lib.macosx-10.9-x86_64-3.7/cooltools/api
copying cooltools/api/saddle.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/api
copying cooltools/api/directionality.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/api
copying cooltools/api/init.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/api
copying cooltools/api/coverage.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/api
copying cooltools/api/insulation.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/api
copying cooltools/api/expected.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/api
copying cooltools/api/sample.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/api
copying cooltools/api/eigdecomp.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/api
copying cooltools/api/dotfinder.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/api
copying cooltools/api/snipping.py -> build/lib.macosx-10.9-x86_64-3.7/cooltools/api
running build_ext
building 'cooltools.lib._numutils' extension
creating build/temp.macosx-10.9-x86_64-3.7
creating build/temp.macosx-10.9-x86_64-3.7/cooltools
creating build/temp.macosx-10.9-x86_64-3.7/cooltools/lib
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/isenhart/opt/anaconda3/envs/pentads/include -arch x86_64 -I/Users/isenhart/opt/anaconda3/envs/pentads/include -arch x86_64 -I/Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages/numpy/core/include -I/Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages/numpy/core/include -I/Users/isenhart/opt/anaconda3/envs/pentads/include/python3.7m -c cooltools/lib/_numutils.c -o build/temp.macosx-10.9-x86_64-3.7/cooltools/lib/_numutils.o
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
error: command 'gcc' failed with exit status 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for cooltools
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [58 lines of output]
[1/5] Cythonizing cytoolz/utils.pyx
[2/5] Cythonizing cytoolz/dicttoolz.pyx
[3/5] Cythonizing cytoolz/functoolz.pyx
[4/5] Cythonizing cytoolz/itertoolz.pyx
[5/5] Cythonizing cytoolz/recipes.pyx
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.9-x86_64-3.7
creating build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/compatibility.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/utils_test.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/_version.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/init.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/_signatures.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
creating build/lib.macosx-10.9-x86_64-3.7/cytoolz/curried
copying cytoolz/curried/operator.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/curried
copying cytoolz/curried/init.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/curried
copying cytoolz/curried/exceptions.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/curried
copying cytoolz/itertoolz.pyx -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/dicttoolz.pyx -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/functoolz.pyx -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/recipes.pyx -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/utils.pyx -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/utils.pxd -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/init.pxd -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/recipes.pxd -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/functoolz.pxd -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/dicttoolz.pxd -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/cpython.pxd -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
copying cytoolz/itertoolz.pxd -> build/lib.macosx-10.9-x86_64-3.7/cytoolz
creating build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_none_safe.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_utils.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_curried.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_compatibility.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_embedded_sigs.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_functoolz.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_inspect_args.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_doctests.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_tlz.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_signatures.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/dev_skip_test.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_recipes.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_docstrings.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_dev_skip_test.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_dicttoolz.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_serialization.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_curried_toolzlike.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
copying cytoolz/tests/test_itertoolz.py -> build/lib.macosx-10.9-x86_64-3.7/cytoolz/tests
running build_ext
building 'cytoolz.dicttoolz' extension
creating build/temp.macosx-10.9-x86_64-3.7
creating build/temp.macosx-10.9-x86_64-3.7/cytoolz
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/isenhart/opt/anaconda3/envs/pentads/include -arch x86_64 -I/Users/isenhart/opt/anaconda3/envs/pentads/include -arch x86_64 -I/Users/isenhart/opt/anaconda3/envs/pentads/include/python3.7m -c cytoolz/dicttoolz.c -o build/temp.macosx-10.9-x86_64-3.7/cytoolz/dicttoolz.o
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
error: command 'gcc' failed with exit status 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for cytoolz
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [12 lines of output]
/Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages/setuptools/dist.py:726: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
% (opt, underscore_opt)
running bdist_wheel
running build
running build_ext
building 'pypairix' extension
creating build
creating build/temp.macosx-10.9-x86_64-3.7
creating build/temp.macosx-10.9-x86_64-3.7/src
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/isenhart/opt/anaconda3/envs/pentads/include -arch x86_64 -I/Users/isenhart/opt/anaconda3/envs/pentads/include -arch x86_64 -D_FILE_OFFSET_BITS=64 -D_USE_KNETFILE=1 -Isrc -I/Users/isenhart/opt/anaconda3/envs/pentads/include/python3.7m -c src/bgzf.c -o build/temp.macosx-10.9-x86_64-3.7/src/bgzf.o
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
error: command 'gcc' failed with exit status 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pypairix
error: subprocess-exited-with-error

× Running setup.py install for pypairix did not run successfully.
│ exit code: 1
╰─> [14 lines of output]
/Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages/setuptools/dist.py:726: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
% (opt, underscore_opt)
running install
/Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
setuptools.SetuptoolsDeprecationWarning,
running build
running build_ext
building 'pypairix' extension
creating build
creating build/temp.macosx-10.9-x86_64-3.7
creating build/temp.macosx-10.9-x86_64-3.7/src
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/isenhart/opt/anaconda3/envs/pentads/include -arch x86_64 -I/Users/isenhart/opt/anaconda3/envs/pentads/include -arch x86_64 -D_FILE_OFFSET_BITS=64 -D_USE_KNETFILE=1 -Isrc -I/Users/isenhart/opt/anaconda3/envs/pentads/include/python3.7m -c src/bgzf.c -o build/temp.macosx-10.9-x86_64-3.7/src/bgzf.o
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
error: command 'gcc' failed with exit status 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> pypairix

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

failed

CondaEnvException: Pip failed

@magnitov
Copy link
Owner

Dear @rlisenhart,

I believe you have some issues with CommandLineTools on your machine. These are the lines describing the error:

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
error: command 'gcc' failed with exit status 1

Quick googling gives some hints:

Can you try to solve this issue and let me know if it works?

Best,
Mikhail

@rlisenhart
Copy link
Author

Thanks so much @magnitov. I think it was a problem setting up a new computer. I have now installed and I can run the test data, but I am still having a problem running my own data.

I'm wondering if the issue is coming from the smaller chromosome segments included in the cool file? I've included two errors below. After getting the first one, I tried to remove the small chromosome segments from the bedGraph compartment signal file since I only care about the main chromosome arms. That seemed to run a bit further and resulted in the second error below. Any advice you have would be greatly appreciated!!

Error 1:
Running cis pentad calculation for ./cool/brown_5kb.cool with ./HOMER_compartments/chrom_PC1_merged_res5kb.PC1.bedGraph.txt
Processing cis data...
Chromosome chrX...
Chromosome chr2L...
Chromosome chr2R...
Chromosome chr3L...
Chromosome chr3R...
Chromosome chr4...
Chromosome chrUn_CP007081v1...
Chromosome chrUn_CP007076v1...
Chromosome chrUn_CP007080v1...
Chromosome chrUn_CP007120v1...
Chromosome chrUn_CP007074v1...
Chromosome chrY_CP007107v1_random...
Chromosome chrY_CP007108v1_random...
Traceback (most recent call last):
File "./pentads/src/get_pentad_cis.py", line 316, in
intervals_A, intervals_B)
File "./pentads/src/get_pentad_cis.py", line 149, in get_area_type
if (interval_1 in intervals_A and interval_2 in intervals_B) or
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Error 2 (after filtering the BedGraph file for only the main chromosomes (X, 2L, 2R, 3L, 3R, Y, 4)).
Running cis pentad calculation for ./cool/brown_5kb.cool with ./HOMER_compartments/extraRemove_PC1_merged_res5kb.PC1.bedGraph.txt
Processing cis data...
Chromosome chrX...
Chromosome chr2L...
Chromosome chr2R...
Chromosome chr3L...
Chromosome chr3R...
Chromosome chr4...
Chromosome chrUn_CP007081v1...
Chromosome chrUn_CP007076v1...
Chromosome chrUn_CP007080v1...
Chromosome chrUn_CP007120v1...
Chromosome chrUn_CP007074v1...
Chromosome chrY_CP007107v1_random...
Chromosome chrY_CP007108v1_random...
Chromosome chrY_CP007109v1_random...
Chromosome chrUn_CP007084v1...
Chromosome chrUn_CP007090v1...
Chromosome chrUn_DS483562v1...
Chromosome chrUn_CP007105v1...
Chromosome chrUn_CP007087v1...
Chromosome chrUn_CP007085v1...
Chromosome chrUn_CP007072v1...
Chromosome chrY_CP007118v1_random...
Chromosome chrUn_CP007098v1...
Chromosome chrY_CP007112v1_random...
Chromosome chrUn_CP007088v1...
Chromosome chrUn_CP007077v1...
Chromosome chrUn_CP007082v1...
Chromosome chrY_CP007111v1_random...
Chromosome chrY_CP007113v1_random...
Chromosome chrX_CP007103v1_random...
Chromosome chrY_CP007110v1_random...
Chromosome chrY_CP007114v1_random...
Chromosome chrUn_CP007094v1...
Chromosome chrUn_CP007092v1...
Chromosome chrUn_DS483705v1...
Chromosome chrX_CP007104v1_random...
Chromosome chrUn_CP007096v1...
Chromosome chrUn_DS483707v1...
Chromosome chrY_CP007116v1_random...
Chromosome chrUn_CP007093v1...
Chromosome chrUn_CP007095v1...
Chromosome chrUn_CP007083v1...
Chromosome chrUn_CP007101v1...
Chromosome chrY_CP007117v1_random...
Chromosome chrUn_CP007079v1...
Chromosome chrUn_CP007086v1...
Chromosome chrUn_CP007078v1...
Chromosome chrY_CP007115v1_random...
Chromosome chrUn_DS483723v1...
Chromosome chrUn_CP007091v1...
Chromosome chrUn_CP007071v1...
Chromosome chrUn_DS483709v1...
Chromosome chrUn_CP007089v1...
Chromosome chrUn_DS483734v1...
Chromosome chrUn_DS483629v1...
Chromosome chrUn_DS483735v1...
Chromosome chrUn_DS483726v1...
Chromosome chrUn_DS483711v1...
Chromosome chrUn_DS483641v1...
Chromosome chrUn_DS483712v1...
Chromosome chrUn_DS483646v1...
Chromosome chrUn_DS483647v1...
Chromosome chrUn_DS483736v1...
Chromosome chrX_DS483648v1_random...
Chromosome chrUn_DS483649v1...
Chromosome chrUn_DS483650v1...
Chromosome chrUn_DS483737v1...
Chromosome chrX_DS483655v1_random...
Chromosome chrUn_DS483724v1...
Chromosome chrUn_CP007097v1...
Chromosome chrUn_DS483658v1...
Chromosome chrUn_DS483659v1...
Chromosome chrX_DS483660v1_random...
Chromosome chrUn_DS483662v1...
Chromosome chrUn_DS483663v1...
Chromosome chrX_DS483665v1_random...
Chromosome chrUn_CP007073v1...
Chromosome chrX_DS483666v1_random...
Chromosome chrY_DS483725v1_random...
Chromosome chrUn_DS483738v1...
Chromosome chrX_DS483669v1_random...
Chromosome chrUn_DS483670v1...
Chromosome chrUn_CP007102v1...
Chromosome chrUn_DS483728v1...
Chromosome chrUn_DS483673v1...
Chromosome chrUn_CP007099v1...
Chromosome chrUn_DS483674v1...
Chromosome chrUn_DS483675v1...
Chromosome chrY_DS483677v1_random...
Chromosome chrUn_DS483739v1...
Chromosome chrUn_DS483678v1...
Chromosome chrUn_DS483679v1...
Chromosome chrUn_DS483680v1...
Chromosome chrUn_DS483681v1...
Chromosome chrUn_DS483682v1...
Chromosome chrX_DS483685v1_random...
Chromosome chrUn_DS483686v1...
Chromosome chrUn_DS483687v1...
Chromosome chrUn_DS483688v1...
Chromosome chrUn_DS483689v1...
Chromosome chrUn_DS483719v1...
Chromosome chrUn_DS483740v1...
Chromosome chrY_DS483690v1_random...
Chromosome chrUn_DS483692v1...
Chromosome chrUn_CP007075v1...
Chromosome chrUn_DS483693v1...
Chromosome chrUn_DS483694v1...
Chromosome chrUn_DS483741v1...
Chromosome chrY_DS483742v1_random...
Chromosome chrUn_DS483695v1...
Chromosome chrUn_DS483743v1...
Chromosome chrX_DS483698v1_random...
Chromosome chrY_CP007119v1_random...
Chromosome chrUn_DS483700v1...
Chromosome chrUn_DS483701v1...
Chromosome chrUn_DS483702v1...
Chromosome chrUn_DS483703v1...
Chromosome chrUn_CP007100v1...
Chromosome chrUn_DS483744v1...
Chromosome chrX_DS483745v1_random...
Chromosome chrUn_DS483746v1...
Chromosome chrUn_DS483748v1...
Chromosome chrUn_DS483749v1...
Chromosome chrUn_DS483750v1...
Chromosome chrUn_DS483751v1...
Chromosome chrUn_DS483753v1...
Chromosome chrUn_DS483754v1...
Chromosome chrUn_DS483755v1...
Chromosome chrUn_DS483757v1...
Chromosome chrUn_DS483758v1...
Chromosome chrUn_DS483759v1...
Chromosome chrUn_DS483760v1...
Chromosome chrUn_DS483762v1...
Chromosome chrUn_DS483763v1...
Chromosome chrUn_DS483767v1...
Chromosome chrUn_DS483768v1...
Chromosome chrUn_DS483769v1...
Chromosome chrUn_DS483770v1...
Chromosome chrUn_DS483772v1...
Chromosome chrUn_DS483773v1...
Chromosome chrUn_DS483774v1...
Chromosome chrUn_DS483776v1...
Chromosome chrY_DS483778v1_random...
Chromosome chrUn_DS483780v1...
Chromosome chrUn_DS483781v1...
Chromosome chrUn_DS483782v1...
Chromosome chrUn_DS483783v1...
Chromosome chrX_DS483784v1_random...
Chromosome chrUn_DS483785v1...
Chromosome chrUn_DS483787v1...
Chromosome chrY_DS483788v1_random...
Chromosome chrX_DS483789v1_random...
Chromosome chrY_DS483790v1_random...
Chromosome chrUn_DS483792v1...
Chromosome chrUn_DS483793v1...
Chromosome chrX_DS483795v1_random...
Chromosome chrUn_DS483796v1...
Chromosome chrUn_DS483797v1...
Chromosome chrUn_DS483798v1...
Chromosome chrUn_DS483799v1...
Chromosome chrUn_DS483800v1...
Chromosome chrUn_DS483801v1...
Chromosome chrX_DS483803v1_random...
Chromosome chrUn_DS483804v1...
Chromosome chrUn_DS483805v1...
Chromosome chrUn_DS483806v1...
Chromosome chrUn_DS483807v1...
Chromosome chrUn_DS483808v1...
Chromosome chrX_DS483809v1_random...
Chromosome chrUn_DS483810v1...
Chromosome chrUn_DS483811v1...
Traceback (most recent call last):
File "./pentads/src/get_pentad_cis.py", line 303, in
matrix, *other = numutils.observed_over_expected(matrix)
File "/Users/isenhart/opt/anaconda3/envs/pentads/lib/python3.7/site-packages/cooltools/lib/numutils.py", line 556, in _logbins_numba
assert data10_int[0] == lo
AssertionError

@magnitov
Copy link
Owner

Hi @rlisenhart,

Good to know that it works.

I assume this is Drosophila data. From my experience, you probably do not have any bins with compartments for all these unplaced scaffolds. For Drosophila data I would restrict the analysis to chr2L, chr2R, chr3L, chr3R, and chrX. That is what most people do.

For now, Pentad only has --excl_chrms parameter, which is definitely not nice in this case, since you have to manually list all of these scaffolds. I will add parameter --incl_chrms tomorrow that will allow choosing chromosomes for the analysis. I will keep you posted.

Best,
Mikhail

@magnitov
Copy link
Owner

Hi @rlisenhart ,

I added parameter --incl_chrms that will allow you to select the desired chromosomes for all Pentad scripts. Can you try again with --incl_chrms 2L,2R,3L,3R,X?

Best,
Mikhail

@rlisenhart
Copy link
Author

Hi @magnitov,

Thanks so much! The addition of the --incl_chrms parameter greatly helps! The cis analysis works nicely, as does the distance cutoff!

The trans analysis is still not running for me. It prints:

Running trans pentad calculation for [cool_file] with [compartment_file]

but then nothing else happens. I left it overnight just to see if it was going slowly, but it doesn't seem to be doing anything. Since there is no error message its hard to figure out what is happening.

Also I have a quick question. Is there a way to directly export the values for each compartment analyzed rather than the average for the whole chromosome? I'd like to look at a distribution of the compartment A strengths in control vs in treatment. I was wondering if that was what you plotted in your box plots of your paper next to the pentads schematics.

-Randi

@magnitov
Copy link
Owner

Hi @rlisenhart,

Great news, I'm glad it works!

Did you get the message "'Trans-expected values calculated!'" ? If not, the calculation of in-trans observed-over-expected is not done. I guess that's where the script gets stuck. Since cooltools does it for the whole matrix, it means all chromosomes are included in this step, despite the specified parameter --incl_chrms. If that is indeed the case, I have to think about how to update this step.

What I plotted in the paper is actually a boxplot of average compartment strength per chromosome. However, for Drosophila it might indeed be not optimal. If you want to have a distribution of compartment A strengths, I think it's best to save all the sub-matrices and then calculate these values manually. I can add a flag that will allow one to dump all sub-matrices into a file. You would have to then make an average AB compartment and use it as denominator for each compartment A area extracted. Does this makes sense to you?

Best,
Mikhail

@rlisenhart
Copy link
Author

Hi @magnitov,

I do not get the message "Trans-expected values calculated!", so I think it is stuck before that. That makes sense that the cooltools would operate on the whole matrix.

I see, yes plotting per chromosome makes sense for organisms with quite a few, but my similar plot looks a bit strange coming from 5 data points. If it is not too much trouble to add the flag to dump the individual matrices, that sounds like it would be ideal.

Thanks again,
Randi

@magnitov
Copy link
Owner

Hi @rlisenhart,

I am really sorry that it took way longer that you could have expected.

I have added flag --save_submatrices that allows you to save all the extracted parts. It is saved into .npy file. You can load it with np.load() for downstream analyses. It consists of 5 arrays of arrays, indexed from 0 to 4 for cis submatrices (0 for A at diagonal, 1 for B at diagonal, 2 for A off-diagonal, 3 for B off-diagonal, and 4 for AB) and from 0 to 2 for trans submatrices (0 for A, 1 for B, and 2 for AB). The number of submatrices are as per saved json.

If you still plan to use it, I would appreciate your feedback.

Best,
Mikhail

@rlisenhart
Copy link
Author

Hi @magnitov ,

Thanks so much for building in that feature. I'd love to try it.

I don't mean to be going back in time here, but I'm actually having trouble just using the tool again. The computer I was running it on before died so I'm using a new machine to try to run it on a computing cluster. I can follow your installation prompts, but when I run the test I get many errors involving the package cv2 (output included below). I had this problem when I set it up before, but with the help of a lab mate we were able to solve the problem by re-installing the opencv package. This is not solving the problem now.

I've gone down some rabbit holes in the past week of uninstalling and re-installing variations of opencv (opencv-python, opencv-python-headless, etc) but nothing seems to work. I don't know if there is a version issue or something, but I can't run anything.

I'm not sure if you have any advice. Other Hi-C packages seem to be running on this computing cluster fine (hicexplorer, pygenometracks, etc) so I don't think its a problem there. It all seems to boil down to this opencv package.

-Randi

@rlisenhart
Copy link
Author

I forgot to include the error. I won't include the whole thing because its long. Basically the wget of the files works, but from there I get this:

compartment_signal.txt.gz 100%[=========================================================================>] 1.38M 3.47MB/s in 0.4s

2022-11-10 12:08:14 (3.47 MB/s) - ‘compartment_signal.txt.gz’ saved [1451622/1451622]

Traceback (most recent call last):
File "/home/isenhart/HiC_files_for_HPC/pentads/./src/get_pentad_cis.py", line 4, in
import cv2
ModuleNotFoundError: No module named 'cv2'
Traceback (most recent call last):
File "/home/isenhart/HiC_files_for_HPC/pentads/./src/get_pentad_trans.py", line 4, in
import cv2
ModuleNotFoundError: No module named 'cv2'
Traceback (most recent call last):
File "/home/isenhart/HiC_files_for_HPC/pentads/./src/get_pentad_distance.py", line 5, in
import cv2
ModuleNotFoundError: No module named 'cv2'
Traceback (most recent call last):
File "/home/isenhart/HiC_files_for_HPC/pentads/./src/get_pentad_cis.py", line 4, in
import cv2
ModuleNotFoundError: No module named 'cv2'
Traceback (most recent call last):
File "/home/isenhart/HiC_files_for_HPC/pentads/./src/get_pentad_trans.py", line 4, in
import cv2

This error keeps repeating for each command trying to run in the test file

@magnitov
Copy link
Owner

@rlisenhart Have you created conda environment for Pentad using the YML file from the repository, or you are setting it up by yourself?

@hanshanmengqi
Copy link

Dear @magnitov,

Thank you a lot for the interesting tool, it's very useful for my project.

But I met a problem that I can't get right results I think.

This is installing info blow.

Successfully installed PyWavelets-1.3.0 asciitree-0.3.3 bioframe-0.3.3 charset-normalizer-3.0.1 click-8.1.3 cooler-0.9.1 cooltools-0.5.1 cytoolz-0.12.1 dill-0.3.6 h5py-3.8.0 idna-3.4 imageio-2.26.0 importlib-metadata-6.0.0 joblib-1.2.0 llvmlite-0.39.1 multiprocess-0.70.14 networkx-2.6.3 numba-0.56.4 pyyaml-6.0 requests-2.28.2 scikit-image-0.19.3 scikit-learn-1.0.2 simplejson-3.18.3 threadpoolctl-3.1.0 tifffile-2021.11.2 toolz-0.12.0 urllib3-1.26.14 zipp-3.15.0
done

To activate this environment, use
$ conda activate pentads-env
To deactivate an active environment, use
$ conda deactivate

And next I ran test script and got results, please find the attached errors txt and results what I got.

Pentad.txt
test.zip

Could you please help me find what happened?

Thank you!
Best,
Han

@magnitov
Copy link
Owner

magnitov commented Mar 6, 2023

Hi @hanshanmengqi,

I believe this is cause by a new version of cooler being released recently. You got v0.9 installed, while I was using v0.8 while developing the tool. I have updated YML file to install correct versions. Can you remove your current pentad-env environment and try creating it with a new YML file?

Best,
Mikhail

@hanshanmengqi
Copy link

Hello @magnitov ,

Thank you for your swift reply.

I tried new YML file but got same problem.

Installing info I got:

Successfully built cytoolz pypairix
Installing collected packages: pypairix, asciitree, zipp, urllib3, toolz, tifffile, threadpoolctl, simplejson, pyyaml, PyWavelets, networkx, llvmlite, joblib, imageio, idna, h5py, dill, charset-normalizer, scikit-learn, scikit-image, requests, multiprocess, importlib-metadata, cytoolz, numba, click, bioframe, cooler, cooltools
Successfully installed PyWavelets-1.3.0 asciitree-0.3.3 bioframe-0.3.3 charset-normalizer-3.1.0 click-8.1.3 cooler-0.8.11 cooltools-0.5.1 cytoolz-0.10.1 dill-0.3.6 h5py-3.8.0 idna-3.4 imageio-2.26.0 importlib-metadata-6.0.0 joblib-1.2.0 llvmlite-0.39.1 multiprocess-0.70.14 networkx-2.6.3 numba-0.56.4 pypairix-0.3.7 pyyaml-6.0 requests-2.28.2 scikit-image-0.19.3 scikit-learn-1.0.2 simplejson-3.18.3 threadpoolctl-3.1.0 tifffile-2021.11.2 toolz-0.12.0 urllib3-1.26.14 zipp-3.15.0

And error txt file:

Pentad.error.txt

Thank you for your help.
Best,
Han

@magnitov
Copy link
Owner

magnitov commented Mar 6, 2023

@hanshanmengqi

Do you still get NaNs instead of matrices in json file? Because that is the main problem - the script does not generate the expected data for plotting. I also did not see the logs printed that are supposed to accompany the execution of the tool. I initially thought that it is caused by a new version of cooler, but it seems to be not the case.

Mikhail

@hanshanmengqi
Copy link

@magnitov

Yes, the json files still got NaNs. Need I check version of each tool/package?
Here is output with new YML file:
pentad_test.zip

Han

@magnitov
Copy link
Owner

magnitov commented Mar 6, 2023

@hanshanmengqi, I tried it as well on a different machine and I can reproduce the issue. I will dive a bit deeper. I'll let you know when I fix it.

@hanshanmengqi
Copy link

@magnitov , Thank you very much.

@magnitov
Copy link
Owner

magnitov commented Mar 6, 2023

@hanshanmengqi, should be fixed now. I had to pull one of the earlier versions and make some changes though. If you were expecting to select specific chromosomes for the analysis, this subset is now not working. I will fix it, but can take a while. Let me know if the tools finally works for you.

@hanshanmengqi
Copy link

@magnitov , It's worked! I ran test data and got normal heatmap! Thank you a lot.

Best,
Han

@hanshanmengqi
Copy link

@magnitov Dear Mikhail,

I tried my data but got error which .json contain NaNs.

Do you mind I sent you my data and check what I should fix?

Thank you in advance.

Best,
Han

@magnitov
Copy link
Owner

magnitov commented Mar 8, 2023

Dear Han @hanshanmengqi ,

Sure, let's try that. You can send me a link by email: m.magnitov@nki.nl.

Best,
Mikhail

@hanshanmengqi
Copy link

@magnitov ,
I've send my data to you by email.
Thank you a lot.

Han

@hanshanmengqi
Copy link

@magnitov
I've got beautiful heatmaps with your help.
Thank you!!!
Han

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants