Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove compressed flag from all mmseqs modules #7211

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

jasmezz
Copy link
Contributor

@jasmezz jasmezz commented Dec 12, 2024

We've been struggling a long time with mmseqs modules which wouldn't run with the GTDB database (in the nf-core/funcscan pipeline in our case). The reproducible error occurred always late in the "prefilter" step of the mmseqs runs with infos like aggregatetaxweights died.

@Darcy220606 recently solved the mystery:

[The flag] --compressed 1 is hardcoded across all mmseqs module on nf-core, that apparently is creating a problem when no contigs pass the prefilter step at a certain kmer size, which is a bit downstream in the general prefilter process (esp. that we have a relatively strict parameters). This confuses the tool as the files are 'emptily compressd'.

Hence, we ran all modules with the --compressed 1 removed and now they run through successfully even with the big GTDB database.
Taking this as confirmation, the flag is removed in all mmseqs modules in this PR. The flag can then be re-added as a parameter on pipeline level if needed.

PR checklist

Closes #XXX

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Emit the versions.yml file.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

maxulysse
maxulysse previously approved these changes Dec 12, 2024
vagkaratzas
vagkaratzas previously approved these changes Dec 12, 2024
@famosab
Copy link
Contributor

famosab commented Dec 13, 2024

Seems like some of the snapshots need to be updated for the tests to pass :)

@SPPearce SPPearce dismissed stale reviews from vagkaratzas and maxulysse via 2dc53e5 December 15, 2024 19:57
@SPPearce
Copy link
Contributor

Ugh, 3 of these modules are still in pytest. Are you able to port them to nf-test?

@famosab
Copy link
Contributor

famosab commented Dec 16, 2024

  • mmseqs/search
  • mmseqs/easysearch

and?

@SPPearce
Copy link
Contributor

  • mmseqs/search
  • mmseqs/easysearch

and?

tsv2exprofiledb

@famosab
Copy link
Contributor

famosab commented Dec 17, 2024

I tried with 2/3 modules and ran into a couple of problems but maybe that can be a starting point for the porting :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants