Feature request: Make it easier to clean work directories

[ewels]

It's well known that Nextflow work directories can get huge and need to be deleted as soon as they're no longer needed.

I find this to be a bit trickier when using Tower as I am slightly removed from the execution of the workflow - especially when using AWS it's kind a of a hassle to have to go off and log into s3, dig around to find the work directories and delete them after each run.

At some point we will get around to setting up automatic lifecycle policies on work directories in s3 etc. But what would be even nicer would be if Nextflow could help with the management of intermediate files. Less work for the end user and it would then be transferrable across different compute systems.

Ideas:

• Button on a workflow to delete intermediate files (basically run nextflow clean)
• Reporting page listing all existing workflow directories of runs, plus their age
• Ability to bulk-delete work directories across multiple workflow runs
• Option to automatically manage and delete these files. For example, a setting in the profile / compute environment to automatically delete the work directory 10 days after the workflow is complete.

Phil

[pditommaso]

Oops was missing this. Yes, this is useful, the crucial point is that this should be done by Tower or finally solved on Nextflow side (nextflow-io/nextflow#452) which would be better tho much more complicated.

[ewels]

Can be both I guess? Short term a simple button would be nice 🙄😅

[pditommaso]

I was waiting for this! 😆