Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for snakemake --batch parameter #325

Closed
holtgrewe opened this issue Jan 9, 2023 · 0 comments · Fixed by #326
Closed

Add support for snakemake --batch parameter #325

holtgrewe opened this issue Jan 9, 2023 · 0 comments · Fixed by #326

Comments

@holtgrewe
Copy link
Member

holtgrewe commented Jan 9, 2023

Is your feature request related to a problem? Please describe.
Running snappy-snake on large workflows is hard. Snakemake offers --batch for this.

Describe the solution you'd like
Pass this through from snappy-snake to snakemake.

Also, adjust the pipeline_job.sh template such that it can be submitted easily with sbatch --array=START-STOP. This should lead to adding --batch default=$SLURM_ARRAY_TASK_ID/$SLURM_ARRAY_TASK_MAX to the script and terminate the script if $SLURM_ARRAY_TASK_ID < 1. This will allow submitting with sbatch --array 1-10 pipeline.sh` and work through the large workflow in 10 batches.

Describe alternatives you've considered
N/A

Additional context
N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant