Cannot remove convert-BAM2FASTQ work directroy: Directory not empty

[zhuchcn]

This is a repeated issue: /hot/project/process/MissingPeptides/MISP-000132-MissingPeptidesPanCanP1/CPTAC_PAAD/WXS/work/d6/18e638e93144f211ed12a14280164c/slurm.out

convert-BAM2FASTQ failed.

[13/11e8f9] process > convert_BAM2FASTQ:extract_r... [100%] 4 of 4 ✔
[a2/9ec20a] process > convert_BAM2FASTQ:create_CS... [100%] 4 of 4 ✔
[0a/fe75c5] process > convert_BAM2FASTQ:call_conv... [100%] 1 of 1, failed: 1
[-        ] process > align_DNA:create_CSV_align_DNA -
[-        ] process > align_DNA:call_align_DNA       -
[-        ] process > recalibrate_BAM:create_YAML... -
[-        ] process > recalibrate_BAM:run_recalib... -
[-        ] process > recalibrate_BAM:run_recalib... -
[-        ] process > generate_SQC_BAM:create_YAM... -
[-        ] process > generate_SQC_BAM:run_genera... -
[-        ] process > call_gSNP:create_YAML_call_... -
[-        ] process > call_gSNP:run_call_gSNP        -
[-        ] process > call_sSNV:create_YAML_call_... -
[-        ] process > call_sSNV:run_call_sSNV        -
[-        ] process > call_mtSNV:create_YAML_call... -
[-        ] process > call_mtSNV:run_call_mtSNV      -
ERROR ~ Error executing process > 'convert_BAM2FASTQ:call_convert_BAM2FASTQ (1)'

But the pipeline actually finished successfully, except that the working directory cannot be removed.

executor >  local (245)
  [e6/34a10b] process > run_validate_PipeVal (1)       [100%] 1 of 1 ✔
  [fb/1d9d94] process > get_bam_stats_SAMtools (1)     [100%] 1 of 1 ✔
  [d8/1bd1c2] process > get_readcount_BAM (1)          [100%] 1 of 1 ✔
  [73/0f8264] process > revert_alignment_Picard_Too... [100%] 1 of 1 ✔
  [35/b5e26f] process > collate_bam_SAMtools (35)      [100%] 48 of 48 ✔
  [76/9f8f37] process > remove_intermediate_files (48) [100%] 48 of 48 ✔
  [34/b966b1] process > create_fastqs_SAMtools (48)    [100%] 48 of 48 ✔
  [b9/6fdfbd] process > get_readcount_FASTQ (96)       [100%] 96 of 96 ✔
  [dd/edb8d1] process > compare_readcounts (1)         [100%] 1 of 1 ✔
  Completed at: 03-Apr-2024 10:39:59
  Duration    : 58m 9s
  CPU hours   : 55.8
  Succeeded   : 245


  rm: cannot remove ‘/hot/project/process/MissingPeptides/MISP-000132-MissingPeptidesPanCanP1/CPTAC_PAAD/WXS/work/d6/18e638e93144f211ed12a14280164c/work/work-bam2fastq-C3L-00017-02/97/55aacf56446fb98c3cd9f14bd61d1e’: Directory not empty

I'm guessing maybe this is because Ln 55 exceuted too early, before the publishDir finishes moving files to the destination directory, because the working directory that it failed to remove is indeed empty now.

metapipeline-DNA/module/convert_BAM2FASTQ/call_convert_BAM2FASTQ.nf

Lines 41 to 55 in 59b9663

	set -euo pipefail
	printf "${params_to_dump}" > combined_bam2fastq_params.yaml
	WORK_DIR=${params.work_dir}/work-bam2fastq-${sample}
	mkdir \$WORK_DIR && chmod 2777 \$WORK_DIR
	nextflow \
	-C ${moduleDir}/default.config \
	run ${moduleDir}/../../external/pipeline-convert-BAM2FASTQ/main.nf \
	-params-file combined_bam2fastq_params.yaml \
	--input_csv ${input_csv} \
	--output_dir \$(pwd) \
	--work_dir \$WORK_DIR
	rm -r \$WORK_DIR

[zhuchcn]

What I don't understand is rm -r should remove the directory even if it is not empty. So why does it complain that the directoy isn't empty? Maybe the files were being used (for example being moved by publishDir)?

[yashpatel6]

The publishDir rules shouldn't have any race conditions since the L55 doesn't run until the nextflow run finishes and that only finished after all publishDir have finished; I'm not sure why the complaint is there about rm -r