Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up output from generate_logs.sh #65

Open
Nick-Eagles opened this issue Jan 8, 2021 · 1 comment
Open

Clean up output from generate_logs.sh #65

Nick-Eagles opened this issue Jan 8, 2021 · 1 comment
Assignees

Comments

@Nick-Eagles
Copy link
Member

Currently, scripts/generate_logs.sh makes the assumption that all nextflow processes have completed (whether successfully or not). However, whenever the pipeline halts before completing (e.g. due to a temporary filesystem issue), some processes are typically killed mid-execution, leading to some files in some working directories being absent.

scripts/generate_logs.sh attempts to proceed, taking quite some time for larger experiments, and filling up the main log (`SPEAQeasy_output.log) with error messages such as:

cat: [insert work directory here]/.exitcode: No such file or directory

Instead, scripts/generate_logs.sh should properly handle missing files, possibly stating they are missing in the per-sample logs.

@Nick-Eagles Nick-Eagles self-assigned this Jan 8, 2021
Nick-Eagles added a commit that referenced this issue Oct 6, 2021
…in output log-- addresses at least one piece of #65
Nick-Eagles added a commit that referenced this issue Aug 5, 2022
…c logs: missing exit code files and missing input directory (which can happen if the pipeline fails to even start). Part of #65
@gpertea
Copy link
Member

gpertea commented Oct 26, 2023

I have just used a quick'n'clumsy patch for this in the jhpce_custom branch, see:
https://github.com/LieberInstitute/SPEAQeasy/blob/jhpce_custom/scripts/generate_logs.sh
.. maybe it could be fetched into master if you think it's acceptable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants