-
Notifications
You must be signed in to change notification settings - Fork 84
Metagenomic raw reads analysis workflow without assembly #1014
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
They are missing DBs on cvmfs (Sylph, Groot and DeepARG) so tests will fail but I ask IDC to added DBs : galaxyproject/idc#77 |
Test Results (powered by Planemo)Test Summary
Errored Tests
|
Test Results (powered by Planemo)Test Summary
Errored Tests
|
Test Results (powered by Planemo)Test Summary
Errored Tests
|
|
Hi, it seems that the tests fail for sylph job due to memory issue, do you know how to handle this ? ❌ Failed job f08065326a26f9cc: |
Test Results (powered by Planemo)Test Summary
Errored Tests
|
Test Results (powered by Planemo)Test Summary
Errored Tests
|
Test Results (powered by Planemo)Test Summary
Errored Tests
|
1 similar comment
Test Results (powered by Planemo)Test Summary
Errored Tests
|
Test Results (powered by Planemo)Test Summary
Errored Tests
|
Test Results (powered by Planemo)Test Summary
Errored Tests
|
|
Hi @mvdbeek, did you have any idea why this tool failed ? In test summary we got this one : |
|
Ran out of memory. I'd check against a live instance how much memory this needs and see if you can optimize anything about the test data ? |
|
I have this metrics linked to memory that I found on europe instance on a previous test : |
|
Can you share the link ? I think allocated and recorded values are probably mixed up ? |
Job details for sylph with the same input : |
|
You need to share the history when you share links, i can't access these otherwise. |
Sorry, here is the shared history : https://usegalaxy.eu/u/lefeuvre-h/h/cwl-target-history |
So this job should have failed because it used more memory than allocated. @bgruening are you not enforcing memory limits ? For the issue at hand, 25GB memory is too much, the fastq files don't look too large, so I guess the memory consumption is due to the reference database. There's another reference database, do you know if this is any smaller ? You could try running against that one. |
There is smaller database linked to this tool but I need to add them into cvmfs, hope it don't take too much time |
|
Request on idc : galaxyproject/idc#77 |
|
/review 1014 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds a new workflow for metagenomic analysis from raw reads without assembly, focused on taxonomic profiling and antimicrobial resistance (AMR) detection. The workflow is contributed by the ABRomics consortium and processes paired-end short reads to provide both taxonomic and functional (AMR) information using multiple tools including Sylph, Groot, and DeepARG, with results normalized through argNorm and aggregated via ToolDistillator.
Key changes:
- Introduces a comprehensive metagenomic analysis workflow combining taxonomic assignment (Sylph/Sylph-tax) and AMR detection (Groot/DeepARG)
- Implements results uniformization through argNorm for standardized AMR gene reporting
- Includes ToolDistillator integration for aggregating and summarizing multi-tool outputs
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| metagenomic-raw-reads-amr-analysis.ga | Main workflow file implementing the analysis pipeline with 24 steps, including quality control, taxonomic profiling, AMR detection, and results aggregation |
| metagenomic-raw-reads-amr-analysis-tests.yml | Test configuration with Zenodo-hosted test data and output assertions for validating workflow functionality |
| README.md | Documentation explaining workflow purpose, processing steps, and input requirements |
| CHANGELOG.md | Version history documenting the initial 1.0 release |
| .dockstore.yml | Dockstore configuration with workflow metadata, author information including ORCID for one author |
workflows/microbiome/metagenomic-raw-reads-amr-analysis/README.md
Outdated
Show resolved
Hide resolved
workflows/microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysis.ga
Outdated
Show resolved
Hide resolved
workflows/microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysis.ga
Outdated
Show resolved
Hide resolved
workflows/microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysis.ga
Outdated
Show resolved
Hide resolved
...s/microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysis-tests.yml
Show resolved
Hide resolved
workflows/microbiome/metagenomic-raw-reads-amr-analysis/README.md
Outdated
Show resolved
Hide resolved
Test Results (powered by Planemo)Test Summary
Errored Tests
|
workflows/microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysis.ga
Outdated
Show resolved
Hide resolved
Co-authored-by: Bérénice Batut <berenice.batut@gmail.com>
Test Results (powered by Planemo)Test Summary
Errored Tests
|
1 similar comment
Test Results (powered by Planemo)Test Summary
Errored Tests
|
…tagenomics raw reads is in development
… from metagenomics raw reads is in development" This reverts commit 77f5ecc.
bebatut
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot @hugolefeuvre!
I made suggestions (mostly to rename inputs/outputs and hide intermediate outputs).
As we already discussed, it would be really useful to have a MultiQC report at the end to have one report aggregating all results.
workflows/microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysis.ga
Outdated
Show resolved
Hide resolved
workflows/microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysis.ga
Outdated
Show resolved
Hide resolved
workflows/microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysis.ga
Outdated
Show resolved
Hide resolved
workflows/microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysis.ga
Outdated
Show resolved
Hide resolved
workflows/microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysis.ga
Outdated
Show resolved
Hide resolved
| "paired-collection(s)", | ||
| "fastq", | ||
| "ABRomics", | ||
| "raw-reads" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| "raw-reads" | |
| "short-reads" |
| } | ||
| }, | ||
| "tags": [ | ||
| "Metagenomics", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| "Metagenomics", | |
| "metagenomics", |
| "tags": [ | ||
| "Metagenomics", | ||
| "paired-collection(s)", | ||
| "fastq", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| "fastq", | |
| "taxonomy", | |
| "Antibiotic Resistance Genes", |
workflows/microbiome/metagenomic-raw-reads-amr-analysis/README.md
Outdated
Show resolved
Hide resolved
workflows/microbiome/metagenomic-raw-reads-amr-analysis/README.md
Outdated
Show resolved
Hide resolved
bebatut
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot @hugolefeuvre!
I made suggestions (mostly to rename inputs/outputs and hide intermediate outputs).
As we already discussed, it would be really useful to have a MultiQC report at the end to have one report aggregating all results.
Co-authored-by: Bérénice Batut <berenice.batut@gmail.com>
Test Results (powered by Planemo)Test Summary
Passed Tests
|
I made the rest of the suggested changes. |
FOR CONTRIBUTOR:
FOR REVIEWERS:
This workflow does/runs/performs … xyz … to generate/analyze/etc …namefield should be human readable (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless generally understood-) over underscore (_), prefer all lowercase. Folder becomes repository in iwc-workflows organization and is included in TRS id