Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Numpy Error appears in Example-Workflow-Execution #39

Closed
KuechlerO opened this issue May 24, 2024 · 6 comments
Closed

Numpy Error appears in Example-Workflow-Execution #39

KuechlerO opened this issue May 24, 2024 · 6 comments

Comments

@KuechlerO
Copy link

KuechlerO commented May 24, 2024 •
edited
Loading

Hey guys, thx very much for your tool!

I just wanted to try it out, because it seems really interesting.
However, I stumbled over the following error when I tried to execute the example workflow:

AttributeError: module 'numpy' has no attribute 'object'.

Here the whole output:

Building DAG of jobs...
Creating conda environment ../envs/environment_spliceai_rocksdb.yaml...
Downloading and installing remote packages.
Environment for /home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/splicing_pred/DNA/../../../envs/environment_spliceai_rocksdb.yaml created (location: .snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_)
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job                     count    min threads    max threads
--------------------  -------  -------------  -------------
absplice_dna                1              1              1
all                         1              1              1
download_human_fasta        1              1              1
download_splicemaps         1              1              1
mmsplice_splicemap          1              1              1
spliceai                    1              1              1
spliceai_vcf_to_csv         1              1              1
total                       7              1              1

Select jobs to execute...

[Fri May 24 12:42:46 2024]
rule download_splicemaps:
    output: ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi3_method=kn_event_filter=median_cutoff.csv.gz, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi5_method=kn_event_filter=median_cutoff.csv.gz
    jobid: 2
    reason: Missing output files: ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi3_method=kn_event_filter=median_cutoff.csv.gz, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi5_method=kn_event_filter=median_cutoff.csv.gz
    wildcards: genome=hg19, tissue=Brain_Cortex
    resources: tmpdir=/tmp

Downloading SpliceMaps...
  0%|                                                                                                                              | 0/98 [00:00<?, ?it/s]../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi5_method=kn_event_filter=median_cutoff.csv.gz
 12%|██████████████▎                                                                                                      | 12/98 [00:02<00:20,  4.26it/s]../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi3_method=kn_event_filter=median_cutoff.csv.gz
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 98/98 [00:06<00:00, 16.23it/s]
[Fri May 24 12:42:54 2024]
Finished job 2.
1 of 7 steps (14%) done
Select jobs to execute...

[Fri May 24 12:42:54 2024]
rule download_human_fasta:
    output: ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    jobid: 1
    reason: Missing output files: ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    resources: tmpdir=/tmp

--2024-05-24 12:42:54--  https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_39/GRCh37_mapping/GRCh37.primary_assembly.genome.fa.gz
Auflösen des Hostnamens proxy.charite.de (proxy.charite.de)… 141.42.5.215
Verbindungsaufbau zu proxy.charite.de (proxy.charite.de)|141.42.5.215|:8080 … verbunden.
Proxy-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
Länge: 869924494 (830M) [application/x-gzip]
Wird in »STDOUT« gespeichert.

-                                       34%[========================>                                                  ] 286,27M  1,99MB/s    ETA-                                      34%[=======================>                                                ] 286,33M  1,9-   -                                  100%[===============================================================>] 829,62M  1,99MB/s    in 7m 3s

2024-05-24 12:49:57 (1,96 MB/s) - auf die Standardausgabe geschrieben [869924494/869924494]

[Fri May 24 12:49:57 2024]
Finished job 1.
2 of 7 steps (29%) done
Select jobs to execute...

[Fri May 24 12:49:57 2024]
rule mmsplice_splicemap:
    input: ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz, ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi5_method=kn_event_filter=median_cutoff.csv.gz, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi3_method=kn_event_filter=median_cutoff.csv.gz
    output: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_MMSplice_SpliceMap.csv
    jobid: 4
    reason: Missing output files: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_MMSplice_SpliceMap.csv; Input files updated by another job: ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi3_method=kn_event_filter=median_cutoff.csv.gz, ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi5_method=kn_event_filter=median_cutoff.csv.gz
    wildcards: genome=hg19, vcf_id=example_hg19.vcf.gz
    resources: tmpdir=/tmp, mem_mb=32000, mem_mib=30518, threads=4

285750it [00:00, 353311.71it/s]
284409it [00:00, 452003.67it/s]
2024-05-24 12:50:25.336222: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
0it [00:00, ?it/s]2024-05-24 12:50:25.684759: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2024-05-24 12:50:25.697128: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400000000 Hz
WARNING:tensorflow:5 out of the last 5 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7f2ad8583310> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
1it [00:00,  1.62it/s]
[Fri May 24 12:50:27 2024]
Finished job 4.
3 of 7 steps (43%) done
Select jobs to execute...

[Fri May 24 12:50:27 2024]
rule spliceai:
    input: ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz, ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    output: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf
    jobid: 6
    reason: Missing output files: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf; Input files updated by another job: ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    wildcards: genome=hg19, vcf_id=example_hg19.vcf.gz
    resources: tmpdir=/tmp, mem_mb=16000, mem_mib=15259, threads=1, gpu=1

Activating conda environment: .snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_
/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/framework/dtypes.py:585: FutureWarning: In the future `np.object` will be defined as the corresponding NumPy scalar.
  np.object,
Traceback (most recent call last):
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/bin/spliceai", line 7, in <module>
    from spliceai.__main__ import main
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/spliceai/__main__.py", line 5, in <module>
    from spliceai.utils import Annotator, get_delta_scores
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/spliceai/utils.py", line 6, in <module>
    from keras.models import load_model
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/keras/__init__.py", line 21, in <module>
    from tensorflow.python import tf2
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/__init__.py", line 46, in <module>
    from tensorflow.python import data
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/data/__init__.py", line 25, in <module>
    from tensorflow.python.data import experimental
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/data/experimental/__init__.py", line 97, in <module>
    from tensorflow.python.data.experimental import service
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/data/experimental/service/__init__.py", line 353, in <module>
    from tensorflow.python.data.experimental.ops.data_service_ops import distribute
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/data/experimental/ops/data_service_ops.py", line 26, in <module>
    from tensorflow.python.data.experimental.ops import compression_ops
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/data/experimental/ops/compression_ops.py", line 20, in <module>
    from tensorflow.python.data.util import structure
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/data/util/structure.py", line 26, in <module>
    from tensorflow.python.data.util import nest
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/data/util/nest.py", line 40, in <module>
    from tensorflow.python.framework import sparse_tensor as _sparse_tensor
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/framework/sparse_tensor.py", line 28, in <module>
    from tensorflow.python.framework import constant_op
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/framework/constant_op.py", line 29, in <module>
    from tensorflow.python.eager import execute
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/eager/execute.py", line 27, in <module>
    from tensorflow.python.framework import dtypes
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/tensorflow/python/framework/dtypes.py", line 585, in <module>
    np.object,
  File "/home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_/lib/python3.9/site-packages/numpy/__init__.py", line 324, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'object'.
`np.object` was a deprecated alias for the builtin `object`. To avoid this error in existing code, use `object` by itself. Doing this will not modify any behavior and is safe.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
[Fri May 24 12:50:27 2024]
Error in rule spliceai:
    jobid: 6
    input: ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz, ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    output: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf
    conda-env: /home/kuechleo/splice-prediction-tools/absplice/absplice/example/workflow/.snakemake/conda/7b529f67ba9587a5b21fa83be7c2e50e_
    shell:
        spliceai -I ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz -O ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf -R ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa -A grch37
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-05-24T123919.336725.snakemake.log

My set-up: I used the set-up optoin where one uses a Conda-Environment.

My Conda-Env:

conda list
# packages in environment at /home/kuechleo/miniforge3/envs/absplice_env:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
abseil-cpp                20210324.0           h9c3ff4c_0    conda-forge
absl-py                   0.15.0                   pypi_0    pypi
absplice                  0.0.1                     dev_0    <develop>
aioeasywebdav             2.4.0           py38h578d9bd_1001    conda-forge
aiohttp                   3.9.5            py38h01eb140_0    conda-forge
aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
amply                     0.1.6              pyhd8ed1ab_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
argcomplete               3.3.0              pyhd8ed1ab_0    conda-forge
argh                      0.31.2             pyhd8ed1ab_0    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
arrow-cpp                 4.0.0           py38ha12db41_1_cpu    conda-forge
astor                     0.8.1              pyh9f0ad1d_0    conda-forge
asttokens                 2.4.1                    pypi_0    pypi
astunparse                1.6.3              pyhd8ed1ab_0    conda-forge
async-timeout             4.0.3              pyhd8ed1ab_0    conda-forge
attmap                    0.13.2             pyhd8ed1ab_0    conda-forge
attrs                     23.2.0             pyh71513ae_0    conda-forge
aws-c-cal                 0.5.11               h95a6274_0    conda-forge
aws-c-common              0.6.2                h7f98852_0    conda-forge
aws-c-event-stream        0.2.7               h3541f99_13    conda-forge
aws-c-io                  0.10.5               hfb6a706_0    conda-forge
aws-checksums             0.1.11               ha31a3da_7    conda-forge
aws-sdk-cpp               1.8.186              hecaee15_4    conda-forge
backcall                  0.2.0                    pypi_0    pypi
bcrypt                    4.1.3            py38h31a4407_0    conda-forge
binaryornot               0.4.4                      py_1    conda-forge
biopython                 1.83             py38h01eb140_0    conda-forge
blinker                   1.8.2              pyhd8ed1ab_0    conda-forge
boto3                     1.34.111           pyhd8ed1ab_0    conda-forge
botocore                  1.34.112        pyge38_1234567_0    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
brotli-python             1.1.0            py38h17151c0_1    conda-forge
bzip2                     1.0.8                hd590300_5    conda-forge
c-ares                    1.28.1               hd590300_0    conda-forge
ca-certificates           2024.2.2             hbcca054_0    conda-forge
cachetools                4.2.4              pyhd8ed1ab_0    conda-forge
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
cffi                      1.14.4           py38ha312104_0    conda-forge
chardet                   5.2.0            py38h578d9bd_1    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
cloudpickle               3.0.0                    pypi_0    pypi
coin-or-cbc               2.10.10              h9002f0b_0    conda-forge
coin-or-cgl               0.60.7               h516709c_0    conda-forge
coin-or-clp               1.17.8               h1ee7a9c_0    conda-forge
coin-or-osi               0.108.10             haf5fa05_0    conda-forge
coin-or-utils             2.11.11              hee58242_0    conda-forge
coincbc                   2.10.10           0_metapackage    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
coloredlogs               15.0.1             pyhd8ed1ab_3    conda-forge
colorlog                  6.8.2            py38h578d9bd_0    conda-forge
comm                      0.2.2                    pypi_0    pypi
commonmark                0.9.1                      py_0    conda-forge
conda                     4.12.0           py38h578d9bd_0    conda-forge
conda-package-handling    2.2.0              pyh38be061_0    conda-forge
conda-package-streaming   0.9.0              pyhd8ed1ab_0    conda-forge
configargparse            1.7                pyhd8ed1ab_0    conda-forge
connection_pool           0.0.3              pyhd3deb0d_0    conda-forge
contourpy                 1.1.1            py38h7f3f72f_1    conda-forge
cookiecutter              2.6.0              pyhca7485f_0    conda-forge
cryptography              39.0.0           py38h1724139_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
cython                    3.0.10           py38h17151c0_0    conda-forge
cyvcf2                    0.30.15                  pypi_0    pypi
dash                      2.10.2                   pypi_0    pypi
dash-core-components      2.0.0                    pypi_0    pypi
dash-cytoscape            1.0.1                    pypi_0    pypi
dash-html-components      2.0.0                    pypi_0    pypi
dash-table                5.0.0                    pypi_0    pypi
dataclasses               0.8                pyhc8e2a94_3    conda-forge
datrie                    0.8.2            py38h01eb140_7    conda-forge
debugpy                   1.8.1                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
deepdiff                  7.0.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
deprecation               2.1.0              pyh9f0ad1d_0    conda-forge
dill                      0.3.8                    pypi_0    pypi
docutils                  0.20.1           py38h578d9bd_3    conda-forge
dpath                     2.1.6              pyha770c72_0    conda-forge
dropbox                   11.36.2            pyhd8ed1ab_0    conda-forge
enum34                    1.1.10           py38h32f6830_2    conda-forge
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
executing                 2.0.1                    pypi_0    pypi
fastbetabino3             0.0.1                    pypi_0    pypi
filechunkio               1.8                        py_2    conda-forge
filelock                  3.14.0             pyhd8ed1ab_0    conda-forge
flask                     2.2.5                    pypi_0    pypi
fonttools                 4.51.0           py38h01eb140_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
frozenlist                1.4.1            py38h01eb140_0    conda-forge
ftputil                   5.1.0              pyhd8ed1ab_0    conda-forge
future                    1.0.0              pyhd8ed1ab_0    conda-forge
gast                      0.3.3                      py_0    conda-forge
gevent                    24.2.1                   pypi_0    pypi
gffutils                  0.13               pyh7cba7a3_0    bioconda
gflags                    2.2.2             he1b5a44_1004    conda-forge
giflib                    5.2.2                hd590300_0    conda-forge
gitdb                     4.0.11             pyhd8ed1ab_0    conda-forge
gitpython                 3.1.43             pyhd8ed1ab_0    conda-forge
glog                      0.4.0                h49b9bf7_3    conda-forge
google-api-core           2.10.0             pyhd8ed1ab_0    conda-forge
google-api-python-client  2.130.0            pyhd8ed1ab_0    conda-forge
google-auth               1.35.0             pyh6c4a22f_0    conda-forge
google-auth-httplib2      0.2.0              pyhd8ed1ab_0    conda-forge
google-auth-oauthlib      0.4.6              pyhd8ed1ab_0    conda-forge
google-cloud-core         2.3.1              pyhd8ed1ab_0    conda-forge
google-cloud-storage      2.11.0             pyh1a96a4e_0    conda-forge
google-crc32c             1.1.2            py38hf9d55a7_5    conda-forge
google-pasta              0.2.0              pyh8c360ce_0    conda-forge
google-resumable-media    2.7.0              pyhd8ed1ab_0    conda-forge
googleapis-common-protos  1.57.0           py38h578d9bd_0    conda-forge
greenlet                  3.0.3                    pypi_0    pypi
grpc-cpp                  1.37.1               h36de60a_0    conda-forge
grpcio                    1.37.1           py38hdd6454d_0    conda-forge
h5py                      2.10.0          nompi_py38h9915d05_106    conda-forge
hdf5                      1.10.6          nompi_h6a2412b_1114    conda-forge
htslib                    1.17                 h6bc39ce_1    bioconda
httplib2                  0.22.0             pyhd8ed1ab_0    conda-forge
humanfriendly             10.0               pyhd8ed1ab_6    conda-forge
icu                       68.2                 h9c3ff4c_0    conda-forge
idna                      3.7                pyhd8ed1ab_0    conda-forge
imageio                   2.34.1                   pypi_0    pypi
importlib-metadata        7.1.0              pyha770c72_0    conda-forge
importlib-resources       6.4.0              pyhd8ed1ab_0    conda-forge
importlib_resources       6.4.0              pyhd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
interpret                 0.2.7                    pypi_0    pypi
interpret-core            0.2.7                    pypi_0    pypi
ipykernel                 6.29.4                   pypi_0    pypi
ipython                   8.12.3                   pypi_0    pypi
itsdangerous              2.2.0                    pypi_0    pypi
jedi                      0.19.1                   pypi_0    pypi
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jmespath                  1.0.1              pyhd8ed1ab_0    conda-forge
joblib                    1.4.2              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h0b41bf4_3    conda-forge
jsonschema                4.22.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
jupyter-client            8.6.2                    pypi_0    pypi
jupyter_core              5.7.2            py38h578d9bd_0    conda-forge
keras                     2.4.3              pyhd8ed1ab_0    conda-forge
keras-preprocessing       1.1.2              pyhd8ed1ab_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kipoi                     0.8.5              pyh5e36f6f_0    bioconda
kipoi-conda               0.1.6                      py_0    bioconda
kipoi-utils               0.7.7              pyh7cba7a3_0    bioconda
kipoiseq                  0.7.1              pyhdfd78af_0    bioconda
kiwisolver                1.4.5            py38h7f3f72f_1    conda-forge
krb5                      1.20.1               hf9c8cef_0    conda-forge
lazy-loader               0.4                      pypi_0    pypi
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.40                 h55db66e_0    conda-forge
libarchive                3.5.1                h3f442fb_1    conda-forge
libblas                   3.9.0           20_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0           20_linux64_openblas    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcurl                   7.87.0               h6312ad2_0    conda-forge
libdeflate                1.13                 h166bdaf_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libevent                  2.1.10               h9b69904_4    conda-forge
libffi                    3.2.1             he1b5a44_1007    conda-forge
libgcc-ng                 13.2.0               h77fa898_7    conda-forge
libgfortran-ng            13.2.0               h69a702a_7    conda-forge
libgfortran5              13.2.0               hca663fb_7    conda-forge
libgomp                   13.2.0               h77fa898_7    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
liblapack                 3.9.0           20_linux64_openblas    conda-forge
liblapacke                3.9.0           20_linux64_openblas    conda-forge
libnghttp2                1.51.0               hdcd2b5c_0    conda-forge
libopenblas               0.3.25          pthreads_h413a1c8_0    conda-forge
libpng                    1.6.43               h2797004_0    conda-forge
libprotobuf               3.15.8               h780b84a_1    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsolv                   0.7.29               ha6fb4c9_0    conda-forge
libsqlite                 3.45.3               h2797004_0    conda-forge
libssh2                   1.10.0               haa6b8db_3    conda-forge
libstdcxx-ng              13.2.0               hc0a3c3a_7    conda-forge
libthrift                 0.14.1               he6d91bd_2    conda-forge
libtiff                   4.2.0                hbd63e13_2    conda-forge
libutf8proc               2.8.0                h166bdaf_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge
libxml2                   2.9.12               h72842e0_0    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
lime                      0.2.0.1                  pypi_0    pypi
llvmlite                  0.41.1                   pypi_0    pypi
logmuse                   0.2.6              pyh8c360ce_0    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
lzo                       2.10              hd590300_1001    conda-forge
mamba                     0.15.3           py38h2aa5da1_0    conda-forge
markdown                  3.6                pyhd8ed1ab_0    conda-forge
markupsafe                2.1.5            py38h01eb140_0    conda-forge
matplotlib-base           3.7.3            py38h58ed7fa_0    conda-forge
matplotlib-inline         0.1.7                    pypi_0    pypi
mmsplice                  2.4.0                    pypi_0    pypi
multidict                 6.0.5            py38h01eb140_0    conda-forge
multiprocess              0.70.16                  pypi_0    pypi
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
natsort                   8.4.0              pyhd8ed1ab_0    conda-forge
nbformat                  5.10.4             pyhd8ed1ab_0    conda-forge
ncls                      0.0.68           py38he5da3d1_2    bioconda
ncurses                   6.5                  h59595ed_0    conda-forge
nest-asyncio              1.6.0                    pypi_0    pypi
networkx                  3.1                      pypi_0    pypi
numba                     0.58.1                   pypi_0    pypi
numpy                     1.23.0           py38h3a7f9d9_0    conda-forge
oauth2client              4.1.3                      py_0    conda-forge
oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
olefile                   0.47               pyhd8ed1ab_0    conda-forge
openjpeg                  2.4.0                hb52868f_1    conda-forge
openssl                   1.1.1w               hd590300_0    conda-forge
opt_einsum                3.3.0              pyhc1e730c_2    conda-forge
orc                       1.6.7                heec2584_1    conda-forge
ordered-set               4.1.0              pyhd8ed1ab_0    conda-forge
orjson                    3.10.3           py38h31a4407_0    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
pandas                    2.0.3            py38h01efb38_1    conda-forge
paramiko                  3.4.0              pyhd8ed1ab_0    conda-forge
parquet-cpp               1.5.1                         2    conda-forge
parso                     0.8.4                    pypi_0    pypi
patsy                     0.5.6              pyhd8ed1ab_0    conda-forge
peppy                     0.40.1             pyhd8ed1ab_0    conda-forge
pexpect                   4.9.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    10.3.0                   pypi_0    pypi
pip                       24.0               pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
plac                      1.4.3              pyhd8ed1ab_0    conda-forge
platformdirs              4.2.2              pyhd8ed1ab_0    conda-forge
plotly                    5.22.0                   pypi_0    pypi
pluggy                    1.5.0              pyhd8ed1ab_0    conda-forge
ply                       3.11               pyhd8ed1ab_2    conda-forge
pooch                     1.8.1              pyhd8ed1ab_0    conda-forge
prettytable               3.10.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.43                   pypi_0    pypi
protobuf                  3.15.8           py38h709712a_0    conda-forge
psutil                    5.9.8            py38h01eb140_0    conda-forge
ptyprocess                0.7.0                    pypi_0    pypi
pulp                      2.7.0            py38h578d9bd_1    conda-forge
pure-eval                 0.2.2                    pypi_0    pypi
pyarrow                   4.0.0           py38hc9229eb_1_cpu    conda-forge
pyasn1                    0.6.0              pyhd8ed1ab_0    conda-forge
pyasn1-modules            0.4.0              pyhd8ed1ab_0    conda-forge
pycosat                   0.6.6            py38h01eb140_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pyfaidx                   0.8.1.1            pyhdfd78af_0    bioconda
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyjwt                     2.8.0              pyhd8ed1ab_1    conda-forge
pynacl                    1.5.0            py38h01eb140_3    conda-forge
pyopenssl                 23.2.0             pyhd8ed1ab_1    conda-forge
pyparsing                 3.1.2              pyhd8ed1ab_0    conda-forge
pyranges                  0.0.125                  pypi_0    pypi
pyrle                     0.0.40           py38he5da3d1_0    bioconda
pysam                     0.21.0           py38h1c8baaf_0    bioconda
pysftp                    0.2.9                      py_1    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pytest                    8.2.1              pyhd8ed1ab_0    conda-forge
python                    3.8.0                h357f687_5    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.19.1             pyhd8ed1ab_0    conda-forge
python-flatbuffers        1.12               pyhd8ed1ab_1    conda-forge
python-irodsclient        2.0.1              pyhd8ed1ab_0    conda-forge
python-slugify            8.0.4              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
python_abi                3.8                      2_cp38    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyu2f                     0.1.5              pyhd8ed1ab_0    conda-forge
pyvcf3                    1.0.3              pyhdfd78af_0    bioconda
pywavelets                1.4.1                    pypi_0    pypi
pyyaml                    6.0.1            py38h01eb140_1    conda-forge
pyzmq                     26.0.3                   pypi_0    pypi
re2                       2021.04.01           h9c3ff4c_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
related                   0.7.3              pyhd8ed1ab_0    conda-forge
reproc                    14.2.4.post0         hd590300_1    conda-forge
reproc-cpp                14.2.4.post0         h59595ed_1    conda-forge
requests                  2.32.2             pyhd8ed1ab_0    conda-forge
requests-oauthlib         2.0.0              pyhd8ed1ab_0    conda-forge
reretry                   0.11.8             pyhd8ed1ab_0    conda-forge
rich                      12.0.1                   pypi_0    pypi
rpds-py                   0.18.1           py38h31a4407_0    conda-forge
rsa                       4.9                pyhd8ed1ab_0    conda-forge
ruamel_yaml               0.15.80         py38h01eb140_1009    conda-forge
s2n                       1.0.10               h9b69904_0    conda-forge
s3transfer                0.10.1             pyhd8ed1ab_0    conda-forge
salib                     1.4.8                    pypi_0    pypi
scikit-image              0.21.0                   pypi_0    pypi
scikit-learn              1.3.2            py38ha25d942_2    conda-forge
scipy                     1.10.1           py38h59b608b_3    conda-forge
seaborn                   0.13.2               hd8ed1ab_2    conda-forge
seaborn-base              0.13.2             pyhd8ed1ab_2    conda-forge
setuptools                70.0.0             pyhd8ed1ab_0    conda-forge
shap                      0.44.1                   pypi_0    pypi
simplejson                3.19.2           py38h01eb140_0    conda-forge
singledispatch            3.6.1              pyh44b312d_0    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
skope-rules               1.0.1                    pypi_0    pypi
slacker                   0.14.0                     py_0    conda-forge
slicer                    0.0.7                    pypi_0    pypi
smart_open                7.0.4              pyhd8ed1ab_0    conda-forge
smmap                     5.0.0              pyhd8ed1ab_0    conda-forge
snakemake                 7.26.0               hdfd78af_0    bioconda
snakemake-minimal         7.26.0             pyhdfd78af_0    bioconda
snappy                    1.1.10               hdb0a2a9_1    conda-forge
sorted_nearest            0.0.39           py38he5da3d1_1    bioconda
spliceai                  1.3.1              pyh864c0ab_1    bioconda
splicemap                 0.0.1                    pypi_0    pypi
sqlite                    3.45.3               h2c6b66d_0    conda-forge
stack-data                0.6.3                    pypi_0    pypi
statsmodels               0.14.1           py38h7f0c24c_0    conda-forge
stone                     3.3.6              pyhd8ed1ab_0    conda-forge
stopit                    1.1.2                      py_0    conda-forge
tabix                     1.11                 hdfd78af_0    bioconda
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
tenacity                  8.3.0                    pypi_0    pypi
tensorboard               2.4.1              pyhd8ed1ab_1    conda-forge
tensorboard-plugin-wit    1.8.1              pyhd8ed1ab_0    conda-forge
tensorflow                2.4.3            py38h578d9bd_0    conda-forge
tensorflow-base           2.4.3            py38h83f5f1d_0    conda-forge
tensorflow-estimator      2.4.0              pyh9656e83_0    conda-forge
termcolor                 1.1.0                    pypi_0    pypi
text-unidecode            1.3                pyhd8ed1ab_1    conda-forge
threadpoolctl             3.5.0              pyhc1e730c_0    conda-forge
throttler                 1.2.2              pyhd8ed1ab_0    conda-forge
tifffile                  2023.7.10                pypi_0    pypi
tinydb                    4.8.0              pyhd8ed1ab_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toposort                  1.10               pyhd8ed1ab_0    conda-forge
tornado                   6.4                      pypi_0    pypi
tqdm                      4.66.4             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
treeinterpreter           0.2.3                    pypi_0    pypi
types-python-dateutil     2.9.0.20240316     pyhd8ed1ab_0    conda-forge
typing-extensions         3.7.4.3                       0    conda-forge
typing_extensions         3.7.4.3                    py_0    conda-forge
ubiquerg                  0.7.0              pyhd8ed1ab_0    conda-forge
unicodedata2              15.1.0           py38h01eb140_0    conda-forge
uritemplate               4.1.1              pyhd8ed1ab_0    conda-forge
urllib3                   1.26.18            pyhd8ed1ab_0    conda-forge
veracitools               0.1.3                      py_0    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
werkzeug                  2.2.3                    pypi_0    pypi
wget                      3.2                      pypi_0    pypi
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
wrapt                     1.12.1                   pypi_0    pypi
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yarl                      1.9.4            py38h01eb140_0    conda-forge
yte                       1.5.4              pyha770c72_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zope-event                5.0                      pypi_0    pypi
zope-interface            6.4.post2                pypi_0    pypi
zstandard                 0.19.0           py38h0a891b7_0    conda-forge
zstd                      1.4.9                ha95c52a_0    conda-forge
@KuechlerO
Copy link
Author

Ah, I've tried also the docker container. However, the same problem appears also in the docker container:

(absplice_dock) root@e39380687b11:/app/absplice/example/workflow# python -m snakemake -j 1 --use-conda
/app/absplice/example/workflow
Building DAG of jobs...
Your conda installation is not configured to use strict channel priorities. This is however crucial for having robust and correct environments (for details, see https://conda-forge.org/docs/user/tipsandtricks.html). Please consider to configure strict priorities by executing 'conda config --set channel_priority strict'.
Creating conda environment ../envs/environment_spliceai_rocksdb.yaml...
Downloading and installing remote packages.
Environment for /app/absplice/example/workflow/splicing_pred/DNA/../../../envs/environment_spliceai_rocksdb.yaml created (location: .snakemake/conda/9241678ce9daadc3d4184a0af19626a0_)
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job                     count    min threads    max threads
--------------------  -------  -------------  -------------
absplice_dna                1              1              1
all                         1              1              1
download_human_fasta        1              1              1
download_splicemaps         1              1              1
mmsplice_splicemap          1              1              1
spliceai                    1              1              1
spliceai_vcf_to_csv         1              1              1
total                       7              1              1

Select jobs to execute...

[Fri May 24 11:08:02 2024]
rule download_splicemaps:
    output: ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi3_method=kn_event_filter=median_cutoff.csv.gz, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi5_method=kn_event_filter=median_cutoff.csv.gz
    jobid: 2
    reason: Missing output files: ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi3_method=kn_event_filter=median_cutoff.csv.gz, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi5_method=kn_event_filter=median_cutoff.csv.gz
    wildcards: genome=hg19, tissue=Brain_Cortex
    resources: tmpdir=/tmp

Downloading SpliceMaps...
  0%|                                                                                                                                                     | 0/98 [00:00<?, ?it/s]../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi5_method=kn_event_filter=median_cutoff.csv.gz
 12%|█████████████████▏                                                                                                                          | 12/98 [00:01<00:08, 10.70it/s]../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi3_method=kn_event_filter=median_cutoff.csv.gz
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 98/98 [00:02<00:00, 48.24it/s]
[Fri May 24 11:08:07 2024]
Finished job 2.
1 of 7 steps (14%) done
Select jobs to execute...

[Fri May 24 11:08:07 2024]
rule download_human_fasta:
    output: ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    jobid: 1
    reason: Missing output files: ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    resources: tmpdir=/tmp

--2024-05-24 11:08:07--  https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_39/GRCh37_mapping/GRCh37.primary_assembly.genome.fa.gz
Resolving ftp.ebi.ac.uk (ftp.ebi.ac.uk)... 193.62.193.165
Connecting to ftp.ebi.ac.uk (ftp.ebi.ac.uk)|193.62.193.165|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 869924494 (830M) [application/x-gzip]
Saving to: ‘STDOUT’

-                                             39%[===================================>                                                        ] 331.22M  1.54MB/s    eta 5m 20s ^-                                             39%[===================================>                                                        ] 331.26M  1.55MB/s    in 3m 33s

2024-05-24 12:03:15 (1.56 MB/s) - Connection closed at byte 347351689. Retrying.

--2024-05-24 12:17:52--  (try: 2)  https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_39/GRCh37_mapping/GRCh37.primary_assembly.genome.fa.gz
Connecting to ftp.ebi.ac.uk (ftp.ebi.ac.uk)|193.62.193.165|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 869924494 (830M), 522572805 (498M) remaining [application/x-gzip]
Saving to: ‘STDOUT’

-                                            100%[++++++++++++++++++++++++++++++++++++=======================================================>] 829.62M  1.80MB/s    in 5m 4s

2024-05-24 12:22:56 (1.64 MB/s) - written to stdout [869924494/869924494]

[Fri May 24 12:22:56 2024]
Finished job 1.
2 of 7 steps (29%) done
Select jobs to execute...

[Fri May 24 12:22:57 2024]
rule mmsplice_splicemap:
    input: ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz, ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi5_method=kn_event_filter=median_cutoff.csv.gz, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi3_method=kn_event_filter=median_cutoff.csv.gz
    output: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_MMSplice_SpliceMap.csv
    jobid: 4
    reason: Missing output files: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_MMSplice_SpliceMap.csv; Input files updated by another job: ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi3_method=kn_event_filter=median_cutoff.csv.gz, ../data/resources/downloaded_files/splicemap_hg19/Brain_Cortex_splicemap_psi5_method=kn_event_filter=median_cutoff.csv.gz
    wildcards: genome=hg19, vcf_id=example_hg19.vcf.gz
    resources: tmpdir=/tmp, mem_mb=32000, mem_mib=30518, threads=4

2024-05-24 12:23:00.140524: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
285750it [00:00, 389305.07it/s]
284409it [00:00, 427808.50it/s]
2024-05-24 12:23:36.082493: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
0it [00:00, ?it/s]2024-05-24 12:23:37.007840: W tensorflow/tsl/platform/profile_utils/cpu_utils.cc:118] Failed to find bogomips or clock in /proc/cpuinfo; cannot determine CPU frequency
WARNING:tensorflow:5 out of the last 5 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fffb09f53a0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
1it [00:01,  1.26s/it]
[Fri May 24 12:23:38 2024]
Finished job 4.
3 of 7 steps (43%) done
Select jobs to execute...

[Fri May 24 12:23:38 2024]
rule spliceai:
    input: ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz, ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    output: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf
    jobid: 6
    reason: Missing output files: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf; Input files updated by another job: ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    wildcards: genome=hg19, vcf_id=example_hg19.vcf.gz
    resources: tmpdir=/tmp, mem_mb=16000, mem_mib=15259, threads=1, gpu=1

Activating conda environment: .snakemake/conda/9241678ce9daadc3d4184a0af19626a0_
/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/framework/dtypes.py:585: FutureWarning: In the future `np.object` will be defined as the corresponding NumPy scalar.
  np.object,
Traceback (most recent call last):
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/bin/spliceai", line 7, in <module>
    from spliceai.__main__ import main
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/spliceai/__main__.py", line 5, in <module>
    from spliceai.utils import Annotator, get_delta_scores
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/spliceai/utils.py", line 6, in <module>
    from keras.models import load_model
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/keras/__init__.py", line 21, in <module>
    from tensorflow.python import tf2
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/__init__.py", line 46, in <module>
    from tensorflow.python import data
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/data/__init__.py", line 25, in <module>
    from tensorflow.python.data import experimental
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/data/experimental/__init__.py", line 97, in <module>
    from tensorflow.python.data.experimental import service
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/data/experimental/service/__init__.py", line 353, in <module>
    from tensorflow.python.data.experimental.ops.data_service_ops import distribute
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/data/experimental/ops/data_service_ops.py", line 26, in <module>
    from tensorflow.python.data.experimental.ops import compression_ops
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/data/experimental/ops/compression_ops.py", line 20, in <module>
    from tensorflow.python.data.util import structure
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/data/util/structure.py", line 26, in <module>
    from tensorflow.python.data.util import nest
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/data/util/nest.py", line 40, in <module>
    from tensorflow.python.framework import sparse_tensor as _sparse_tensor
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/framework/sparse_tensor.py", line 28, in <module>
    from tensorflow.python.framework import constant_op
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/framework/constant_op.py", line 29, in <module>
    from tensorflow.python.eager import execute
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/eager/execute.py", line 27, in <module>
    from tensorflow.python.framework import dtypes
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/tensorflow/python/framework/dtypes.py", line 585, in <module>
    np.object,
  File "/app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_/lib/python3.9/site-packages/numpy/__init__.py", line 324, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'object'.
`np.object` was a deprecated alias for the builtin `object`. To avoid this error in existing code, use `object` by itself. Doing this will not modify any behavior and is safe.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
[Fri May 24 12:23:40 2024]
Error in rule spliceai:
    jobid: 6
    input: ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz, ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    output: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf
    conda-env: /app/absplice/example/workflow/.snakemake/conda/9241678ce9daadc3d4184a0af19626a0_
    shell:
        spliceai -I ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz -O ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf -R ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa -A grch37
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-05-24T110424.018017.snakemake.log

@KuechlerO KuechlerO mentioned this issue May 24, 2024
Open
@neverov-am
Copy link
Collaborator

Dear @KuechlerO ,

Thank you for your interest in our tool and for the very helpful fix suggestion!

We will run some tests and update the environment and the container accordingly.

@yunkaig
Copy link

yunkaig commented Sep 9, 2024

I am experiencing version issues with NumPy.

Building DAG of jobs...
Your conda installation is not configured to use strict channel priorities. This is however crucial for having robust and correct environments (for details, see https://conda-forge.org/docs/user/tipsandtricks.html). Please consider to configure strict priorities by executing 'conda config --set channel_priority strict'.
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job                    count    min threads    max threads
-------------------  -------  -------------  -------------
absplice_dna               1              1              1
all                        1              1              1
spliceai                   1              1              1
spliceai_vcf_to_csv        1              1              1
total                      4              1              1

Select jobs to execute...

[Mon Sep  9 02:02:17 2024]
rule spliceai:
    input: ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz, ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    output: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf
    jobid: 6
    reason: Missing output files: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf
    wildcards: genome=hg19, vcf_id=example_hg19.vcf.gz
    resources: tmpdir=/tmp, mem_mb=16000, mem_mib=15259, threads=1, gpu=1

Activating conda environment: .snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.2 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/bin/spliceai", line 7, in <module>
    from spliceai.__main__ import main
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/spliceai/__main__.py", line 5, in <module>
    from spliceai.utils import Annotator, get_delta_scores
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/spliceai/utils.py", line 2, in <module>
    import pandas as pd
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/pandas/__init__.py", line 26, in <module>
    from pandas.compat import (
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/pandas/compat/__init__.py", line 27, in <module>
    from pandas.compat.pyarrow import (
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/pandas/compat/pyarrow.py", line 8, in <module>
    import pyarrow as pa
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/pyarrow/__init__.py", line 63, in <module>
    import pyarrow.lib as _lib
AttributeError: _ARRAY_API not found

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.2 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/bin/spliceai", line 7, in <module>
    from spliceai.__main__ import main
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/spliceai/__main__.py", line 5, in <module>
    from spliceai.utils import Annotator, get_delta_scores
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/spliceai/utils.py", line 2, in <module>
    import pandas as pd
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/pandas/__init__.py", line 49, in <module>
    from pandas.core.api import (
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/pandas/core/api.py", line 9, in <module>
    from pandas.core.dtypes.dtypes import (
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/pandas/core/dtypes/dtypes.py", line 24, in <module>
    from pandas._libs import (
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/pyarrow/__init__.py", line 63, in <module>
    import pyarrow.lib as _lib
AttributeError: _ARRAY_API not found

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.2 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/bin/spliceai", line 7, in <module>
    from spliceai.__main__ import main
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/spliceai/__main__.py", line 5, in <module>
    from spliceai.utils import Annotator, get_delta_scores
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/spliceai/utils.py", line 6, in <module>
    from keras.models import load_model
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/keras/__init__.py", line 21, in <module>
    from tensorflow.python import tf2
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/__init__.py", line 41, in <module>
    from tensorflow.python.eager import context
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/eager/context.py", line 37, in <module>
    from tensorflow.python.client import pywrap_tf_session
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/client/pywrap_tf_session.py", line 23, in <module>
    from tensorflow.python.client._pywrap_tf_session import *
AttributeError: _ARRAY_API not found

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.2 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/bin/spliceai", line 7, in <module>
    from spliceai.__main__ import main
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/spliceai/__main__.py", line 5, in <module>
    from spliceai.utils import Annotator, get_delta_scores
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/spliceai/utils.py", line 6, in <module>
    from keras.models import load_model
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/keras/__init__.py", line 21, in <module>
    from tensorflow.python import tf2
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/__init__.py", line 46, in <module>
    from tensorflow.python import data
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/__init__.py", line 25, in <module>
    from tensorflow.python.data import experimental
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/experimental/__init__.py", line 97, in <module>
    from tensorflow.python.data.experimental import service
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/experimental/service/__init__.py", line 353, in <module>
    from tensorflow.python.data.experimental.ops.data_service_ops import distribute
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/experimental/ops/data_service_ops.py", line 26, in <module>
    from tensorflow.python.data.experimental.ops import compression_ops
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/experimental/ops/compression_ops.py", line 20, in <module>
    from tensorflow.python.data.util import structure
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/util/structure.py", line 26, in <module>
    from tensorflow.python.data.util import nest
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/util/nest.py", line 40, in <module>
    from tensorflow.python.framework import sparse_tensor as _sparse_tensor
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/framework/sparse_tensor.py", line 28, in <module>
    from tensorflow.python.framework import constant_op
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/framework/constant_op.py", line 29, in <module>
    from tensorflow.python.eager import execute
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/eager/execute.py", line 27, in <module>
    from tensorflow.python.framework import dtypes
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/framework/dtypes.py", line 30, in <module>
    from tensorflow.python.lib.core import _pywrap_bfloat16
AttributeError: _ARRAY_API not found
ImportError: numpy.core._multiarray_umath failed to import
ImportError: numpy.core.umath failed to import
Traceback (most recent call last):
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/bin/spliceai", line 7, in <module>
    from spliceai.__main__ import main
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/spliceai/__main__.py", line 5, in <module>
    from spliceai.utils import Annotator, get_delta_scores
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/spliceai/utils.py", line 6, in <module>
    from keras.models import load_model
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/keras/__init__.py", line 21, in <module>
    from tensorflow.python import tf2
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/__init__.py", line 46, in <module>
    from tensorflow.python import data
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/__init__.py", line 25, in <module>
    from tensorflow.python.data import experimental
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/experimental/__init__.py", line 97, in <module>
    from tensorflow.python.data.experimental import service
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/experimental/service/__init__.py", line 353, in <module>
    from tensorflow.python.data.experimental.ops.data_service_ops import distribute
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/experimental/ops/data_service_ops.py", line 26, in <module>
    from tensorflow.python.data.experimental.ops import compression_ops
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/experimental/ops/compression_ops.py", line 20, in <module>
    from tensorflow.python.data.util import structure
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/util/structure.py", line 26, in <module>
    from tensorflow.python.data.util import nest
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/data/util/nest.py", line 40, in <module>
    from tensorflow.python.framework import sparse_tensor as _sparse_tensor
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/framework/sparse_tensor.py", line 28, in <module>
    from tensorflow.python.framework import constant_op
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/framework/constant_op.py", line 29, in <module>
    from tensorflow.python.eager import execute
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/eager/execute.py", line 27, in <module>
    from tensorflow.python.framework import dtypes
  File "/g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_/lib/python3.9/site-packages/tensorflow/python/framework/dtypes.py", line 33, in <module>
    _np_bfloat16 = _pywrap_bfloat16.TF_bfloat16_type()
TypeError: Unable to convert function return value to a Python type! The signature was
        () -> handle
[Mon Sep  9 02:02:22 2024]
Error in rule spliceai:
    jobid: 6
    input: ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz, ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    output: ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf
    conda-env: /g/data/jb96/yungao/absplice/singularity_home/absplice/example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_
    shell:
        spliceai -I ../data/resources/analysis_files/vcf_files/example_hg19.vcf.gz -O ../data/results/hg19/model_scores_from_absplice_features/example_hg19.vcf.gz_SpliceAI.vcf -R ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa -A grch37
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-09-09T020212.846528.snakemake.log

Apparently there is a numpy==2.0.2 in the conda environment located at example/workflow/.snakemake/conda/cfdf2e3035ba1a7872992ad444be188c_ that is incompatible with spliceai. Trying to resolve this locally but haven't yet found a way not to disrupt the whole environment.

@neverov-am
Copy link
Collaborator

Dear @yunkaig ,

Thank you for reporting this.

We fixed the versions of several packages and in our tests it worked. Could you replace contents of the environment_spliceai_rocksdb.yaml file with the following and let us know if it resolves the issue for you?

name: spliceai-rocksdb
channels:
  - conda-forge
dependencies:
  - python==3.9
  - python-rocksdb>=0.7
  - rocksdb
  - tensorflow-gpu==2.6.2
  - cudnn
  - cudatoolkit==11.0.3
  - pyarrow
  - tqdm
  - pip
  - setuptools
  - click
  - pooch
  - bioconda::snakemake>=6.1
  - bioconda::kipoiseq
  - bioconda::spliceai
  - bioconda::cyvcf2
  - pip:
    - git+https://github.com/gagneurlab/spliceai_rocksdb.git@master
    - numpy==1.19.5
    - pandas==1.4.4

@yunkaig
Copy link

yunkaig commented Sep 10, 2024

Hi @neverov-am,

Thank you for getting back so quickly. I have tried using the new YAML and the example workflow was able to run to completion.

To give you more feedbacks, two very minor points:

  1. The rule download_human_fasta was somehow now able to run due to an issue with certificate. It gave me the following message:
--2024-09-10 09:57:24--  https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_39/GRCh37_mapping/GRCh37.primary_assembly.genome.fa.gz
Resolving ftp.ebi.ac.uk (ftp.ebi.ac.uk)... 193.62.193.165
Connecting to ftp.ebi.ac.uk (ftp.ebi.ac.uk)|193.62.193.165|:443... connected.
ERROR: cannot verify ftp.ebi.ac.uk's certificate, issued by ‘CN=GEANT OV RSA CA 4,O=GEANT Vereniging,C=NL’:
  Self-signed certificate encountered.
To connect to ftp.ebi.ac.uk insecurely, use `--no-check-certificate'.

gzip: stdin: unexpected end of file
[Tue Sep 10 09:57:24 2024]
Error in rule download_human_fasta:
    jobid: 1
    output: ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
    shell:
        wget -O - https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_39/GRCh37_mapping/GRCh37.primary_assembly.genome.fa.gz | gunzip -c > ../data/resources/downloaded_files/GRCh37.primary_assembly.genome.fa
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

The error message was clear enough so it was an easy fix for me.

  1. I guess the absplice python package was not automatically shipped with the container image? There was an error raised for missing module absplice so I realised I still need to run pip install -e . even for containers. So maybe an update of the README?

PS. My facility does not allow root access so I had to go with singularity for the containerised option, but I believe the process should not be much different once set up.

Many thanks,
Yunkai

@neverov-am
Copy link
Collaborator

Hi Yunkai,

Thank you very much for your feedback, it is very helpful for us to know.

We opted for not preinstalling AbSplice inside the container for a couple of reasons, one of them being to make sure users have the latest updates from GitHub as they install the package. Thanks for pointing out that the installation command is missing in the README, we will add it.

Best,
Aleksandr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@yunkaig @WagnerNils @KuechlerO @neverov-am