Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set mkfastq reports and stats outputs to optional #6932

Merged
merged 2 commits into from
Nov 4, 2024

Conversation

alanmmobbs93
Copy link
Contributor

PR checklist

Closes mkfastq fails on AWS Batch #281

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!

Description of error reported:

cellranger/mkfastq process fails when running nf-core/demultiplex on AWS Batch with -profile test_mkfastq,docker.
When running locally, the output folders are created (but without files) hence the process ends succesfully. When running on AWS Batch the process fails with the following error:

Missing output file(s) `*_outs/outs/fastq_path/Reports` expected by process `NFCORE_DEMULTIPLEX:DEMULTIPLEX:MKFASTQ_DEMULTIPLEX:CELLRANGER_MKFASTQ (test_sample.1)`

Some explanation may come from the following AWS doc explaining that S3 is a flat system where files are grouped by common prefixes to simulate folders. It also explains that the S3 folders behave differently than local systems. So, uploading an empty folder to S3 may be the cause of the error reported by @KallyopeComp.

Changes

  • Set reports output to optional: true.
  • Set stats output to optional: true.

Notes:

  • Module snapshot was also tested.
  • Module linting throws a warning related to the container version.

@nschcolnicov nschcolnicov self-requested a review November 4, 2024 15:12
Copy link
Contributor

@nschcolnicov nschcolnicov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@nschcolnicov nschcolnicov added this pull request to the merge queue Nov 4, 2024
Merged via the queue into nf-core:master with commit f7d2d43 Nov 4, 2024
11 checks passed
@alanmmobbs93 alanmmobbs93 deleted the fix/cellranger_mkfastq branch November 4, 2024 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

mkfastq fails on AWS Batch
2 participants