OSError: Duplicate node name "func_preproc_rest_wf" found

[kathy-snider]

Describe the bug
A clear and concise description of what the bug is.
Am a newbie to nibabies (and to nipreps in general). Just trying to get all of the right arguments to process a subject.
Got an OSError from the pipeline. Stack trace is below.

To Reproduce
What specific command did you run?
sudo /opt/acc/sbin/exadocker run --rm -v /mnt/scratch/fnl_lab/680fef771afe429b/BIDS_unprocessed/:/bids_input:ro -v /mnt/scratch/fnl_lab/680fef771afe429b/BIDS_processed/:/output -v /mnt/scratch/fnl_lab/680fef771afe429b/Templates/:/templates -v /mnt/scratch/fnl_lab/680fef771afe429b/license.txt:/opt/freesurfer/license.txt mgxd/nibabies:0.1.0 /bids_input /output participant --participant-label 06IND001B001 --age-months 12 --segmentation-atlases-dir /templates/JLF_templates --skip-bids-validation --nprocs=10 --use-syn-sdc

What version of nibabies did you run? 0.1.0

Expected behavior
A clear and concise description of what you expected to happen.
Expected nibabies to run and to process the subject specfied.

Screenshots
Is it a visual data problem?
If applicable, add screenshots to help explain your problem. N/A.

Additional context
Add any other context about the problem here.
This was run using Docker on a slurm node. From the slurm log:
...

• sudo /opt/acc/sbin/exadocker run --rm -v /mnt/scratch/fnl_lab/680fef771afe429b/BIDS_unprocessed/:/bids_input:ro -v /mnt/scratch/fnl_lab/680fef771afe429b/BIDS_processed/:/output -v /mnt/scratch/fnl_lab/680fef771afe429b/Templates/:/templates -v /mnt/scratch/fnl_lab/680fef771afe429b/license.txt:/opt/freesurfer/license.txt mgxd/nibabies:0.1.0 /bids_input /output participant --participant-label 06IND001B001 --age-months 12 --segmentation-atlases-dir /templates/JLF_templates --skip-bids-validation --nprocs=10 --use-syn-sdc
  Process Process-2:
  Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
  self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
  self._target(*self._args, **self._kwargs)
  File "/src/nibabies/nibabies/cli/workflow.py", line 82, in build_workflow
  retval["workflow"] = init_nibabies_wf()
  File "/src/nibabies/nibabies/workflows/base.py", line 68, in init_nibabies_wf
  single_subject_wf = init_single_subject_wf(subject_id)
  File "/src/nibabies/nibabies/workflows/base.py", line 462, in init_single_subject_wf
  ('outputnode.fsnative2anat_xfm', 'inputnode.fsnative2anat_xfm'),
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/workflows.py", line 158, in connect
  self._check_nodes(newnodes)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/workflows.py", line 766, in _check_nodes
  raise IOError('Duplicate node name "%s" found.' % node.name)
  OSError: Duplicate node name "func_preproc_rest_wf" found.
  ...

[mgxd]

This has come up a few times in fMRIPrep, it is probably one of these things:

• That subject contains both uncompressed and compressed NIfTIs
  • I would check the rest runs for that subject, and ensure they are all different
• A working directory used on a different subject is being reused

[effigies]

This seems like something we should probably detect and give a useful error on...

[oesteban]

* A working directory used on a different subject is being reused

How this could lead to this problem?

* That subject contains both uncompressed and compressed NIfTIs
  * I would check the rest runs for that subject, and ensure they are all different

I would think this is the reason.

@kathy-snider - What's the content of the func/ folder(s) for this particular subject?

ls  /mnt/scratch/fnl_lab/680fef771afe429b/BIDS_unprocessed/sub-06IND001B001/func  /mnt/scratch/fnl_lab/680fef771afe429b/BIDS_unprocessed/sub-06IND001B001/ses-*/func

[kathy-snider]

Actually, this might have yet a different cause, but I'm not sure what it could be.

Each subject is processed on a slurm node with its own scratch space, so it is not really possible for it to use some other job's workspace.
All of the niftis are compressed.

Note: while examining the niftis for compression, I noticed that the functional nifti files did not have BIDS-compliant file names. (They were named with subject_rest-bold.nii.gz rather than subject_task-rest_bold.nii.gz).

I fixed the names of the files. But got the same error (with the new name reflected in the error).

Then, wondered if it mattered that these subjects have no sessions. I didn't think that would matter because sessions are not required under BIDS, but tried putting in a session for each one anyway.

Still got the same error. The final error after these changes was:
OSError: Duplicate node name "func_preproc_ses_001_task_rest_wf" found.

[kathy-snider]

The whole tree for the data, after the changes noted above is:

├── dataset_description.json
├── sub-06IND001B001
│   ├── ses-001
│   │   ├── anat
│   │   │   ├── sub-06IND001B001_ses-001_T1w.json
│   │   │   ├── sub-06IND001B001_ses-001_T1w.nii.gz
│   │   │   ├── sub-06IND001B001_ses-001_T2w.json
│   │   │   └── sub-06IND001B001_ses-001_T2w.nii.gz
│   │   └── func
│   │       ├── sub-06IND001B001_ses-001_task-rest_bold.json
│   │       └── sub-06IND001B001_ses-001_task-rest_bold.nii.gz
│   └── sub-06IND001B001_details.tsv
├── sub-06IND001B002
│   ├── ses-001
│   │   ├── anat
│   │   │   ├── sub-06IND001B002_ses-001_T1w.json
│   │   │   ├── sub-06IND001B002_ses-001_T1w.nii.gz
│   │   │   ├── sub-06IND001B002_ses-001_T2w.json
│   │   │   └── sub-06IND001B002_ses-001_T2w.nii.gz
│   │   └── func
│   │       ├── sub-06IND001B002_ses-001_task-rest_bold.json
│   │       └── sub-06IND001B002_ses-001_task-rest_bold.nii.gz
│   └── sub-06IND001B002_details.tsv
└── sub-06IND001B003
    ├── ses-001
    │   ├── anat
    │   │   ├── sub-06IND001B003_ses-001_T1w.json
    │   │   ├── sub-06IND001B003_ses-001_T1w.nii.gz
    │   │   ├── sub-06IND001B003_ses-001_T2w.json
    │   │   └── sub-06IND001B003_ses-001_T2w.nii.gz
    │   └── func
    │       ├── sub-06IND001B003_ses-001_task-rest_bold.json
    │       └── sub-06IND001B003_ses-001_task-rest_bold.nii.gz
    └── sub-06IND001B003_details.tsv

[mgxd]

Release 0.1.1 is out now, and has BIDS validation included.

[kathy-snider]

Note that the output file had these messages:

210630-17:04:15,886 nipype.workflow IMPORTANT:

Running nibabies version 0.1.0:
  * BIDS dataset path: /bids_input.
  * Participant list: ['06IND001B001'].
  * Run identifier: 20210630-170411_d3a4fb38-396c-4a26-bea1-30e61fd9da8d.
  * Output spaces: None.
  * Pre-run FreeSurfer's SUBJECTS_DIR: /output/freesurfer-infant.

210630-17:04:17,507 nipype.workflow INFO:
No single-band-reference found for sub-06IND001B001_ses-001_task-rest_bold.nii.gz.
210630-17:04:17,612 nipype.workflow INFO:
No single-band-reference found for sub-06IND001B001_ses-001_task-rest_bold.nii.gz.

[mgxd]

your tree has a sub-06IND001B003 folder within the sub-06IND001B002 folder.

Nvm? the formatting is throwing me off

[kathy-snider]

Ran with 0.1.1:
sudo /opt/acc/sbin/exadocker run --rm -v /mnt/scratch/fnl_lab/e555ff7b852c04bd/BIDS_unprocessed/:/bids_input:ro -v /mnt/scratch/fnl_lab/e555ff7b852c04bd/BIDS_processed/:/output -v /mnt/scratch/fnl_lab/e555ff7b852c04bd/Templates/:/templates -v /mnt/scratch/fnl_lab/e555ff7b852c04bd/license.txt:/opt/freesurfer/license.txt mgxd/nibabies:0.1.1 /bids_input /output participant --participant-label 06IND001B003 --age-months 24 --segmentation-atlases-dir /templates/JLF_templates --skip-bids-validation --nprocs=10 --use-syn-sdc

Same error. :(

[mgxd]

Can you remove --skip-bids-validation

[kathy-snider]

New command:
/usr/bin/sbatch -A fnl_lab --job-name NB_TEST --partition exacloud --gres disk:300 --cpus-per-task 20 --mem-per-cpu 15G --time 36:00:00 --output /home/exacloud/gscratch/InspireLab/data/HCP/logs/NB_TEST/NB_TEST/sub-06IND001B001/sub-06IND001B001.out --error /home/exacloud/gscratch/InspireLab/data/HCP/logs/NB_TEST/NB_TEST/sub-06IND001B001/sub-06IND001B001.err --exclude=exanode-6-21,exanode-6-23,exanode-6-27,exanode-6-36,exanode-6-42,exanode-6-7,exanode-7-5 /home/exacloud/gscratch/InspireLab/projects/sniderk/nibabies/submit_to_nibabies.sh sub-06IND001B001 NB_TEST 12 /home/exacloud/gscratch/InspireLab/projects/sniderk/nibabies/niftis --use-syn-sdc

Same result:
OSError: Duplicate node name "func_preproc_ses_001_task_rest_wf" found.

[kathy-snider]

If it helps, here is the stdout stuff:

bids-validator@1.5.7

    1: [WARN] You should define 'SliceTiming' for this file. If you don't provide this information slice time correction will not be possible. (code: 13 - SLICE_TIMING_NOT_DEFINED)
            ./sub-06IND001B001/ses-001/func/sub-06IND001B001_ses-001_task-rest_bold.nii.gz

    Please visit https://neurostars.org/search?q=SLICE_TIMING_NOT_DEFINED for existing conversations about this issue.

    2: [WARN] NIfTI file's header field for unit information for x, y, z, and t dimensions empty or too short (code: 41 - NIFTI_UNIT)
            ./sub-06IND001B001/ses-001/anat/sub-06IND001B001_ses-001_T2w.nii.gz

    Please visit https://neurostars.org/search?q=NIFTI_UNIT for existing conversations about this issue.

    3: [WARN] The recommended file /README is missing. See Section 03 (Modality agnostic files) of the BIDS specification. (code: 101 - README_FILE_MISSING)

    Please visit https://neurostars.org/search?q=README_FILE_MISSING for existing conversations about this issue.

    4: [WARN] The Authors field of dataset_description.json should contain an array of fields - with one author per field. This was triggered because there are no authors, which will make DOI registration from dataset metadata impossible. (code: 113 - NO_AUTHORS)

    Please visit https://neurostars.org/search?q=NO_AUTHORS for existing conversations about this issue.


    Summary:                 Available Tasks:        Available Modalities:
    7 Files, 231.35MB                                T1w
    1 - Subject                                      T2w
    1 - Session                                      bold


    If you have any questions, please post on https://neurostars.org/tags/bids.

210630-17:53:35,833 nipype.workflow IMPORTANT:

Running nibabies version 0.1.0+22.g66577f7:
  * BIDS dataset path: /bids_input.
  * Participant list: ['06IND001B001'].
  * Run identifier: 20210630-175328_8a2b456e-1f68-4e3e-988f-57abc1d8ddc6.
  * Output spaces: None.
  * Pre-run FreeSurfer's SUBJECTS_DIR: /output/freesurfer-infant.

210630-17:53:37,537 nipype.workflow INFO:
No single-band-reference found for sub-06IND001B001_ses-001_task-rest_bold.nii.gz.
210630-17:53:37,643 nipype.workflow INFO:
No single-band-reference found for sub-06IND001B001_ses-001_task-rest_bold.nii.gz.

[mgxd]

would it be possible to share the data directory?