Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

error - eaglec.scoreUtils Not found #19

Open
abhijitcbio opened this issue Mar 24, 2023 · 9 comments
Open

error - eaglec.scoreUtils Not found #19

abhijitcbio opened this issue Mar 24, 2023 · 9 comments

Comments

@abhijitcbio
Copy link

Hi,

I have installed eaglec using pip (python 3.7) option pip install --user eagleC and tried running it on my HiC data. But got the following error

predictSV --hic-5k ./mcools_files/PASMXM.mcool::/resolutions/5000 --hic-10k ./mcools_files/PASMXM.mcool::/resolutions/10000 --hic-50k ./mcools_files/PASMXM.mcool::/resolutions/50000 -O PASMXM -g hg38 --balance-type Raw --output-format full
root                      INFO    @ 03/24/23 16:03:54:
# ARGUMENT LIST:
# Cool URI at 5kb = ./mcools_files/PASMXM.mcool::/resolutions/5000
# Cool URI at 10kb = ./mcools_files/PASMXM.mcool::/resolutions/10000
# Cool URI at 50kb = ./mcools_files/PASMXM.mcool::/resolutions/50000
# Balance Type = Raw
# Reference Genome = hg38
# Included Chromosomes = ['#', 'X']
# Probability Cutoff for 5kb SVs = 0.8
# Probability Cutoff for 10kb SVs = 0.8
# Probability Cutoff for 50kb SVs = 0.99999
# Output File Prefix = PASMXM
# Output Format = full
# Log file name = PASMXM.log
root                      INFO    @ 03/24/23 16:03:54: Predict SVs at 5kb resolution ...
Traceback (most recent call last):
  File "/home/abhijit/.local/bin/predictSV-single-resolution", line 276, in <module>
    run()
  File "/home/abhijit/.local/bin/predictSV-single-resolution", line 110, in run
    from eaglec.scoreUtils import intraPredict, interPredict
ModuleNotFoundError: No module named 'eaglec.scoreUtils'
Traceback (most recent call last):
  File "/home/abhijit/.local/bin/predictSV", line 176, in <module>
    run()
  File "/home/abhijit/.local/bin/predictSV", line 112, in run
    subprocess.check_call(' '.join(command), shell=True)
  File "/home/abhijit/miniconda3/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'predictSV-single-resolution -H ./mcools_files/PASMXM.mcool::/resolutions/5000 --balance-type Raw -O PASMXM.CNN_SVs.5K.txt --genome hg38 --output-format full -C "#" "X" --prob-cutoff 0.8 --logFile PASMXM.log' returned non-zero exit status 1.

I can import eaglec and eaglec.utilities but not eaglec.scoreUtils in python3.7 environment

$python3.7
Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:53)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import eaglec
>>> import eaglec.utilities
>>> import eaglec.scoreUtils
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'eaglec.scoreUtils'
>>> print(eaglec.__file__)
/home/abhijit/.local/lib/python3.7/site-packages/eaglec/__init__.py

Your help is greatly appreciated.
@XiaoTaoWang
Copy link
Owner

Could you upgrade your python version to 3.8 and try again?

@yfarjoun
Copy link

yfarjoun commented Mar 25, 2023 •
edited
Loading

I'm having the exact same issue (on mac):

$ python
Python 3.8.16 | packaged by conda-forge | (default, Feb  1 2023, 16:05:36) 
[Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import eaglec
>>> import eaglec.utilities
>>> import eaglec.scoreUtils
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'eaglec.scoreUtils'
>>>

@yfarjoun
Copy link

yfarjoun commented Mar 25, 2023 •
edited
Loading

A fresh install following the instructions (except that I had to remove python=3.8 from the install command but I was installing into python 3.8) on a AWS linux machine leads to the following:

root                      INFO    @ 03/25/23 04:09:39: 
# ARGUMENT LIST:
# Cool URI at 5kb = SKNAS-MboI-allReps-filtered.mcool::/resolutions/5000
# Cool URI at 10kb = SKNAS-MboI-allReps-filtered.mcool::/resolutions/10000
# Cool URI at 50kb = SKNAS-MboI-allReps-filtered.mcool::/resolutions/50000
# Balance Type = CNV
# Reference Genome = hg38
# Included Chromosomes = ['#', 'X']
# Probability Cutoff for 5kb SVs = 0.8
# Probability Cutoff for 10kb SVs = 0.8
# Probability Cutoff for 50kb SVs = 0.99999
# Output File Prefix = SK-N-AS
# Output Format = full
# Log file name = SK-N-AS.log
root                      INFO    @ 03/25/23 04:09:39: Predict SVs at 5kb resolution ...
Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/EagleC/bin/predictSV-single-resolution", line 276, in <module>
    run()
  File "/home/ec2-user/miniconda3/envs/EagleC/bin/predictSV-single-resolution", line 110, in run
    from eaglec.scoreUtils import intraPredict, interPredict
  File "eaglec/scoreUtils.pyx", line 11, in init eaglec.scoreUtils
  File "/home/ec2-user/miniconda3/envs/EagleC/lib/python3.8/site-packages/eaglec/utilities.py", line 9, in <module>
    from cooler import ice
  File "/home/ec2-user/miniconda3/envs/EagleC/lib/python3.8/site-packages/cooler/__init__.py", line 12, in <module>
    from . import balance, create, fileops, parallel, tools
  File "/home/ec2-user/miniconda3/envs/EagleC/lib/python3.8/site-packages/cooler/balance.py", line 8, in <module>
    from .parallel import partition, split
  File "/home/ec2-user/miniconda3/envs/EagleC/lib/python3.8/site-packages/cooler/parallel.py", line 10, in <module>
    from .core import get
  File "/home/ec2-user/miniconda3/envs/EagleC/lib/python3.8/site-packages/cooler/core/__init__.py", line 9, in <module>
    from ._tableops import delete, get, put
  File "/home/ec2-user/miniconda3/envs/EagleC/lib/python3.8/site-packages/cooler/core/_tableops.py", line 1, in <module>
    import h5py
  File "/home/ec2-user/miniconda3/envs/EagleC/lib/python3.8/site-packages/h5py/__init__.py", line 46, in <module>
    from ._conv import register_converters as _register_converters
  File "h5py/_conv.pyx", line 1, in init h5py._conv
  File "h5py/h5t.pyx", line 293, in init h5py.h5t
  File "/home/ec2-user/miniconda3/envs/EagleC/lib/python3.8/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'typeDict'
Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/EagleC/bin/predictSV", line 176, in <module>
    run()
  File "/home/ec2-user/miniconda3/envs/EagleC/bin/predictSV", line 112, in run
    subprocess.check_call(' '.join(command), shell=True)
  File "/home/ec2-user/miniconda3/envs/EagleC/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'predictSV-single-resolution -H SKNAS-MboI-allReps-filtered.mcool::/resolutions/5000 --balance-type CNV -O SK-N-AS.CNN_SVs.5K.txt --genome hg38 --output-format full -C "#" "X" --prob-cutoff 0.8 --logFile SK-N-AS.log' returned non-zero exit status 1.

I resolved it by updating numpy:

pip install numpy==1.21

@yfarjoun
Copy link

Thanks for the quick responses @XiaoTaoWang! Note that the error above was on linux. The error on mac is still not resolved....any ideas what's going on there?

@XiaoTaoWang
Copy link
Owner

Hi, thanks so much for the feedback! I guess that the error occurred due to a discrepancy between the compiler I used and the one on your Mac. Did you install EagleC on the M1/M2 chip? My Mac is still based on the old intel chip, which may be the reason for the difference.

@yfarjoun
Copy link

Indeed I have the ARM chip, however, I did install everything using the i386 emulator and so it "should" work...I'm also happy to install from source. Where is the source-code hosted?

@abhijitcbio
Copy link
Author

I was able to resolve the issue after switching to the 3.8 version.
Thanks a lot.

@DittmanC
Copy link

I am using python3.8 and numpy==1.21, but no sure what was going wrong...
predictSV --hic-5k SKNAS-MboI-allReps-filtered.mcool::/resolutions/5000
--hic-10k SKNAS-MboI-allReps-filtered.mcool::/resolutions/10000
--hic-50k SKNAS-MboI-allReps-filtered.mcool::/resolutions/50000
-O SK-N-AS -g hg38 --balance-type ICE --output-format full
--prob-cutoff-5k 0.8 --prob-cutoff-10k 0.8 --prob-cutoff-50k 0.99999

root INFO @ 05/18/23 15:05:31:

ARGUMENT LIST:

Cool URI at 5kb = SKNAS-MboI-allReps-filtered.mcool::/resolutions/5000

Cool URI at 10kb = SKNAS-MboI-allReps-filtered.mcool::/resolutions/10000

Cool URI at 50kb = SKNAS-MboI-allReps-filtered.mcool::/resolutions/50000

Balance Type = ICE

Reference Genome = hg38

Included Chromosomes = ['#', 'X']

Probability Cutoff for 5kb SVs = 0.8

Probability Cutoff for 10kb SVs = 0.8

Probability Cutoff for 50kb SVs = 0.99999

Output File Prefix = SK-N-AS

Output Format = full

Log file name = SK-N-AS.log

root INFO @ 05/18/23 15:05:31: Predict SVs at 5kb resolution ...
root INFO @ 05/18/23 15:05:32: matched sequencing depth in human at 10Kb: 80181600.06574439
root INFO @ 05/18/23 15:05:32: Load CNN models from /disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages/eaglec/data/bulk/50M-100M ...
2023-05-18 15:05:32.863496: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-05-18 15:05:32.869113: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
Traceback (most recent call last):
File "/disk3/users/dittman/.conda/envs/py38/bin/predictSV-single-resolution", line 276, in
run()
File "/disk3/users/dittman/.conda/envs/py38/bin/predictSV-single-resolution", line 150, in run
cnn_models.append(create_cnn(cnn_weights))
File "eaglec/CNN.pyx", line 83, in eaglec.CNN.create_cnn
File "/disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 2319, in load_weights
with h5py.File(filepath, 'r') as f:
File "/disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages/h5py/_hl/files.py", line 424, in init
fid = make_fid(name, mode, userblock_size,
File "/disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages/h5py/_hl/files.py", line 190, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 96, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = '/disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages/eaglec/data/bulk/50M-100M/CNN-weights.0.1.0.4.0.6.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
Traceback (most recent call last):
File "/disk3/users/dittman/.conda/envs/py38/bin/predictSV", line 176, in
run()
File "/disk3/users/dittman/.conda/envs/py38/bin/predictSV", line 112, in run
subprocess.check_call(' '.join(command), shell=True)
File "/disk3/users/dittman/.conda/envs/py38/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'predictSV-single-resolution -H SKNAS-MboI-allReps-filtered.mcool::/resolutions/5000 --balance-type ICE -O SK-N-AS.CNN_SVs.5K.txt --genome hg38 --output-format full -C "#" "X" --prob-cutoff 0.8 --logFile SK-N-AS.log' returned non-zero exit status 1.
(py38) dittman@wd1:/Data$ pip install numpy==1.21
Requirement already satisfied: numpy==1.21 in /disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages (1.21.0)
(py38) dittman@wd1:/Data$ predictSV --hic-5k SKNAS-MboI-allReps-filtered.mcool::/resolutions/5000 --hic-10k SKNAS-MboI-allReps-filtered.mcool::/resolutions/10000 --hic-50k SKNAS-MboI-allReps-filtered.mcool::/resolutions/50000 -O SK-N-AS -g hg38 --balance-type ICE --output-format full --prob-cutoff-5k 0.8 --prob-cutoff-10k 0.8 --prob-cutoff-50k 0.99999
root INFO @ 05/18/23 15:07:22:

ARGUMENT LIST:

Cool URI at 5kb = SKNAS-MboI-allReps-filtered.mcool::/resolutions/5000

Cool URI at 10kb = SKNAS-MboI-allReps-filtered.mcool::/resolutions/10000

Cool URI at 50kb = SKNAS-MboI-allReps-filtered.mcool::/resolutions/50000

Balance Type = ICE

Reference Genome = hg38

Included Chromosomes = ['#', 'X']

Probability Cutoff for 5kb SVs = 0.8

Probability Cutoff for 10kb SVs = 0.8

Probability Cutoff for 50kb SVs = 0.99999

Output File Prefix = SK-N-AS

Output Format = full

Log file name = SK-N-AS.log

root INFO @ 05/18/23 15:07:22: Predict SVs at 5kb resolution ...
root INFO @ 05/18/23 15:07:23: matched sequencing depth in human at 10Kb: 80181600.06574439
root INFO @ 05/18/23 15:07:23: Load CNN models from /disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages/eaglec/data/bulk/50M-100M ...
2023-05-18 15:07:23.751928: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-05-18 15:07:23.756636: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
Traceback (most recent call last):
File "/disk3/users/dittman/.conda/envs/py38/bin/predictSV-single-resolution", line 276, in
run()
File "/disk3/users/dittman/.conda/envs/py38/bin/predictSV-single-resolution", line 150, in run
cnn_models.append(create_cnn(cnn_weights))
File "eaglec/CNN.pyx", line 83, in eaglec.CNN.create_cnn
File "/disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 2319, in load_weights
with h5py.File(filepath, 'r') as f:
File "/disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages/h5py/_hl/files.py", line 424, in init
fid = make_fid(name, mode, userblock_size,
File "/disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages/h5py/_hl/files.py", line 190, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 96, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = '/disk3/users/dittman/.conda/envs/py38/lib/python3.8/site-packages/eaglec/data/bulk/50M-100M/CNN-weights.0.1.0.4.0.6.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
Traceback (most recent call last):
File "/disk3/users/dittman/.conda/envs/py38/bin/predictSV", line 176, in
run()
File "/disk3/users/dittman/.conda/envs/py38/bin/predictSV", line 112, in run
subprocess.check_call(' '.join(command), shell=True)
File "/disk3/users/dittman/.conda/envs/py38/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'predictSV-single-resolution -H SKNAS-MboI-allReps-filtered.mcool::/resolutions/5000 --balance-type ICE -O SK-N-AS.CNN_SVs.5K.txt --genome hg38 --output-format full -C "#" "X" --prob-cutoff 0.8 --logFile SK-N-AS.log' returned non-zero exit status 1.

@XiaoTaoWang
Copy link
Owner

Sorry for the late response. Have you downloaded the pre-trained models using download-pretrained-models?

@XiaoTaoWang XiaoTaoWang mentioned this issue Sep 7, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@yfarjoun @XiaoTaoWang @abhijitcbio @DittmanC