Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get error when creating conda environments #18

Open
wangmhan opened this issue Mar 14, 2024 · 0 comments
Open

get error when creating conda environments #18

wangmhan opened this issue Mar 14, 2024 · 0 comments

Comments

@wangmhan
Copy link

Hi, Thanks for the nice pipeline!

I tried to set up the workflow on a big server. I managed to use mamba to set the environment and run smoothly with the test code: sh setup_test.sh. Then I get stuck when runing this code: snakemake --use-conda --cores $Ncores --configfile config/config.yml
I modified it by specify another profile: snakemake --profile ~/profiles/server-slurm --use-conda --cores 1 --configfile config/config.yml

Because the cluster nodes have no internet access, so can only run it on a login node with 1 core. Here is the profile settings:
default-resources: mem_mb=6000

cluster: "submit.py"
cluster-status: "check_status.sh"
cluster-cancel: "scancel"

max-jobs-per-second: 5
max-status-checks-per-second: 0.2
local-cores: 1
latency-wait: 120
jobs: 50
keep-going: True
rerun-incomplete: True
printshellcmds: True

Here is the log:
Workflow defines that rule bwaindex is eligible for caching between workflows (use the --cache argument to enable this).
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cluster nodes: 50
Job stats:
job count min threads max threads


bin_pairs_library 8 1 1
chunk_fastq 4 1 1
default 1 1 1
download_fastqs 1 1 1
map_chunks_bwa_pipe 4 1 1
merge_dedup 4 1 1
merge_runs 5 1 1
merge_stats_libraries_into_groups 3 1 1
merge_zoom_library_group_coolers 6 1 1
parse_sort_chunks 4 1 1
scaling_pairs_library 4 1 1
zoom_library 8 1 1
total 52 1 1

Select jobs to execute...
WorkflowError:
Error grouping resources in group '744b017c-b3d8-4ed6-8e0a-ee3979e6939e': Not enough resources were provided. This error is typically caused by a Pipe group requiring too many resources. Note that resources are summed across every member of the pipe group, except for ['runtime'], which is calculated via max(). Excess Resources:
_cores: 2/1

Do you have any ideas how to solve it? Big thanks!

Best,
Menghan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant