[REVIEW]: QuaC: A Pipeline Implementing Quality Control Best Practices for Genome Sequencing and Exome Sequencing Data #5313
Comments
editorialbot
added
Jinja
review
Shell
TeX
Track: 2 (BCM)
|
Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks. For a list of things I can do to help you, just type: For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type: |
|
|
Wordcount for |
|
|
From @Redmar-van-den-Berg: |
|
@Redmar-van-den-Berg Thanks for accepting to review our pipeline and providing us an initial feedback.
We chose this setup deliberately as this acts as a sanity check for us to be certain that the output and log directories were chosen correctly by the user.
Good point. I have now automated retrieval and processing of the necessary datasets. Docs are updated to reflect this change, and workflow-config is now updated to utilize the datasets downloaded using above script. PS- Please be sure to use |
I don't think this adds a lot of safety, and it does make it more difficult to run the pipeline.
The |
We make output of this script readily available in the repo. Please see |
|
Please note that, once the requirements and configs listed here are satisfied, one should be able to run system testing without any further setup. Please let me know if you run into any issues :) |
Review checklist for @brentpConflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
|
Maybe we are talking about two different things. The file |
QuaC includes script src/setup_dependency_datasets.sh to fetch all the datasets listed under I cleaned up the docs a bit for clarity on this topic. I hope this helps. |
|
Thanks for pointing me to the script, I was able to download the required reference files! However, I run into a crash when I try to run the pipeline itself, following along to the documentation. python src/run_quac.py --project_name test_project --projects_path ".test/ngs-data/" --pedigree ".test/configs/${PRIOR_QC_STATUS}/${PROJECT_CONFIG}.ped" --outdir "$USER_SCRATCH/tmp/quac/results/test_${PROJECT_CONFIG}_exome-${PRIOR_QC_STATUS}/analysis" --quac_watch_config "configs/quac_watch/exome_quac_watch_config.yaml" --exome $USE_SLURM -e="--conda-create-envs-only"
########################################
Command to run the pipeline:
snakemake \
--snakefile '/home/username/devel/quac/workflow/Snakefile' \
--config project_name='test_project' projects_path='/home/username/devel/quac/.test/ngs-data' ped='/home/username/devel/quac/.test/configs/no_priorQC/project_2samples.ped' quac_watch_config='/home/username/devel/quac/configs/quac_watch/exome_quac_watch_config.yaml' workflow_config='/home/username/devel/quac/configs/workflow.yaml' unique_id='2ad58efa-8557-4807-b85d-ed1d4752ff2f' out_dir='/home/username/devel/quac/scratch/tmp/quac/results/test_project_2samples_exome-no_priorQC/analysis' log_dir='/home/username/devel/quac/scratch/tmp/quac/logs' exome='True' include_prior_qc_data='False' allow_sample_renaming='False' \
--restart-times 1 \
--use-conda \
--use-singularity \
--singularity-args '--cleanenv --bind /home/username/devel/quac/scratch/tmp/quac/tmp:/tmp --bind /home/username/devel/quac/.test/ngs-data/test_project/analysis,/home/username/devel/quac/scratch/tmp/quac/logs,/home/username/devel/quac/scratch/tmp/quac/results/test_project_2samples_exome-no_priorQC/analysis,data/external/dependency_datasets/somalier,data/external/dependency_datasets/verifyBamID,data/external/dependency_datasets/reference_genome,/home/username/devel/quac/.test/configs/no_priorQC,/home/username/devel/quac/configs/quac_watch/exome_quac_watch_config.yaml,data/external/dependency_datasets/verifyBamID/exome' \
--profile '/home/username/devel/quac/src/slurm/slurm_profile' \
--conda-create-envs-only
########################################
Slurm job name : "quac-2023-04-05T10.10.34.732874"
Slurm job script : "/home/username/devel/quac/scratch/tmp/quac/logs/quac-2023-04-05T10.10.34.732874.sh"
// Processing project: test_project
// Project path: "/home/username/devel/quac/.test/ngs-data/test_project/analysis"
// Exome mode: True
// Include prior QC data: False
// WARNING: '.test' present in the path supplied via --projects_path. So testing mode is used.
Building DAG of jobs...
Pulling singularity image docker://continuumio/miniconda3:4.7.12.
Pulling singularity image docker://brentp/somalier:v0.2.13.
Creating conda environment configs/env/quac_watch.yaml...
Downloading and installing remote packages.
CreateCondaEnvironmentException:
Could not create conda environment from /home/username/devel/quac/configs/env/quac_watch.yaml:
FATAL: container creation failed: unable to add /home/username/devel/quac/data/external/dependency_datasets/somalier to mount list: destination must be an absolute path
File "/home/username/miniconda3/envs/quac/lib/python3.6/site-packages/snakemake/deployment/conda.py", line 389, in create
Traceback (most recent call last):
File "src/run_quac.py", line 372, in <module>
main(ARGS)
File "src/run_quac.py", line 202, in main
submit_slurm_job(pipeline_cmd, job_dict)
File "/home/username/devel/quac/src/slurm/submit_slurm_job.py", line 60, in submit_slurm_job
job_id = slurm_job.run(**params_dict)
File "/home/username/miniconda3/envs/quac/lib/python3.6/site-packages/slurmpy/slurmpy.py", line 171, in run
res = subprocess.check_output(args).strip()
File "/home/username/miniconda3/envs/quac/lib/python3.6/subprocess.py", line 356, in check_output
**kwargs).stdout
File "/home/username/miniconda3/envs/quac/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bash', '/home/username/devel/quac/scratch/tmp/quac/logs/quac-2023-04-05T10.10.34.732874.sh']' returned non-zero exit status 1. |
|
@Redmar-van-den-Berg I created an issue (uab-cgds-worthey/quac#68) to troubleshoot this error. I hope this works for you. |
We have now refactored QuaC to auto-create these folders (uab-cgds-worthey/quac#67). Please see here for updated docs - https://quac.readthedocs.io/en/latest/ |
|
@Redmar-van-den-Berg @brentp I would like to inform that we have refactored QuaC to move away from creation of conda environment within singularity containers. QuaC now simply depends on webserver-hosted container images, retrieves them automatically as part of the pipeline and uses them for snakemake-initiated jobs (uab-cgds-worthey/quac#70). |
|
Hi @brentp, @Redmar-van-den-Berg, could you give an update on the current status of your review. I saw some work done but it is unclear how much work is left. Thanks |
|
@lpantano I'm still having issues running the pipeline, so I haven't been able to review the pipeline any further. |
|
@ManavalanG would you show how to install the software and run on a fresh system? If possible, I also suggest that you have someone else install and run the example without any guidance from you, perhaps while you watch. Please do this outside of your usual cluster. You have spent much time documenting and building this, but I also am not able to get it running. Currently, I see: It's not clear to me how to resolve this. |
|
@editorialbot generate pdf |
Post-Review Checklist for Editor and AuthorsAdditional Author Tasks After Review is Complete
Editor Tasks Prior to Acceptance
|
|
@editorialbot check references |
|
|
|
@lpantano Zenodo is going through a major upgrade, and it is not working fully as expected atm. QuaC's repo is already part of zenodo, but it doesn't let me update it to the recent QuaC version 1.6. As soon as zenodo is fully working again, I will update the version and will follow up here. Sorry about the delay! |
|
@lpantano I have created a new release 1.6 and archived it in Zenodo (DOI - https://doi.org/10.5281/zenodo.10002036). With this, I believe I have completed all the necessary tasks. Please let me know if I missed any :) |
|
@editorialbot set v1.6 as version |
|
Done! version is now v1.6 |
|
@editorialbot set 10.5281/zenodo.10002036 as archive |
|
Done! archive is now 10.5281/zenodo.10002036 |
|
@editorialbot recommend-accept |
|
|
|
👋 @openjournals/bcm-eics, this paper is ready to be accepted and published. Check final proof 👉📄 Download article If the paper PDF and the deposit XML files look good in openjournals/joss-papers#4707, then you can now move forward with accepting the submission by compiling again with the command |
editorialbot
added
the
recommend-accept
|
@editorialbot set 1.6 as version |
|
Done! version is now 1.6 |
|
@ManavalanG As EiC on this track I have the honor of helping to process the last steps. I have just checked your repository, this review, the paper, and the archive link, and all seems in order. I will now proceed to accept this work. |
|
@editorialbot accept |
|
|
Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository. If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file. You can copy the contents for your CITATION.cff file here: CITATION.cff
If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation. |
|
🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘 |
|
🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨 Here's what you must now do:
Any issues? Notify your editorial technical team... |
|
@ManavalanG congratulations on this JOSS publication! @lpantano thanks for editing! And a special thanks to the reviewers: @brentp, @Redmar-van-den-Berg 🎉 |
|
🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉 If you would like to include a link to your paper from your README use the following code snippets: This is how it will look in your documentation: We need your help! The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:
|
Submitting author: @ManavalanG (Manavalan Gajapathy)
Repository: https://github.com/uab-cgds-worthey/quac
Branch with paper.md (empty if default branch): joss_manuscript
Version: 1.6
Editor: @lpantano
Reviewers: @brentp, @Redmar-van-den-Berg
Archive: 10.5281/zenodo.10002036
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@brentp & @Redmar-van-den-Berg, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @lpantano know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Checklists
📝 Checklist for @brentp
📝 Checklist for @Redmar-van-den-Berg
The text was updated successfully, but these errors were encountered: