Error running CBF pipeline

[willtack]

Describe the bug
Trying to run the cbf pipeline. It prints an error at the very beginning of the run and produces an output folder of log files and mostly empty folders, jsons, and csvs, etc.

Cohort file

id0,img,m0,anatdir
sub-P68,/home/will/data/xcp_testing/SPIRAL_V20_HCP_852307506.nii.gz,/home/will/data/xcp_testing/SPIRAL_V20_HCP_M0_Eq_1.nii.gz,fmriprep/sub-P68/anat

Design File

#!/usr/bin/env bash

###################################################################
#  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  #
###################################################################


###################################################################
# This design file stores the values of all variables required to
# execute a complete neuroimage processing pipeline. You may
# execute the analysis specified in this design file by calling
# (in any v4 or higher bash terminal):
#
# xcpEngine <file>.dsn
#
# Variables fall into five general categories:
# * ANALYSIS VARIABLES are used at all stages of this analysis.
# * PIPELINE specifies the modules that comprise the analysis.
# * MODULE VARIABLES are used during one stage of the analysis.
#                  These are typically array variables with array
#                  indices equal to the index of the module that
#                  calls them.
# * OUTPUT VARIABLES may be used at all stages of the analysis.
#                  These are typically array variables with array
#                  indices equal to the value of the primary
#                  subject identifier. They will appear only in
#                  localised design files.
###################################################################


###################################################################
# ANALYSIS VARIABLES
###################################################################

analysis=cbf_$(whoami)
design=
sequence=cbf
standard=MNI%2x2x2

###################################################################
# PIPELINE
###################################################################

pipeline=prestats,coreg,cbf,roiquant,norm
###################################################################
# 1 PRESTATS
###################################################################

prestats_rerun[1]=1
prestats_cleanup[1]=1
prestats_process[1]=MPR-MCO-ASL

###################################################################
# 2 COREG
###################################################################

coreg_reference[2]=exemplar
coreg_cfunc[2]=bbr
coreg_target[2]=head
coreg_seg[2]=${segmentation[sub]}
coreg_wm[2]=3
coreg_denoise[2]=1
coreg_refwt[2]=NULL
coreg_inwt[2]=NULL
coreg_qacut[2]=0.8,0.9,0.7,0.8
coreg_decide[2]=1
coreg_mask[2]=0
coreg_rerun[2]=0
coreg_cleanup[2]=1

###################################################################
# 3 cbf
###################################################################
cbf_first_tagged[3]=1
cbf_perfusion[3]=casl
cbf_m0_scale[3]=1
cbf_lambda[3]=0.9
cbf_pld[3]=1.8
cbf_tau[3]=1.8
cbf_t1blood[3]=1.65
cbf_alpha[3]=0.72
cbf_gm_val[3]=2,4
cbf_rerun[3]=0
cbf_cleanup[3]=1

###################################################################
# 4 ROIQUANT
###################################################################

roiquant_atlas[4]=all
roiqunat_globals[4]=1
roiquant_vol[4]=0
roiquant_rerun[4]=0
roiquant_cleanup[4]=1

###################################################################
# 5 NORM
###################################################################

norm_rerun[5]=1
norm_cleanup[5]=1

Error message

xcpengine-docker -d /home/will/data/xcp_testing/cbf.dsn -c /home/will/data/xcp_testing/cohort.csv -r /home/will/data/xcp_testing/ -i /home/will/data/xcp_testing/work -o /home/will/data/xcp_testing/output


['/home/will/.local/bin/xcpengine-docker', '-d', '/home/will/data/xcp_testing/cbf.dsn', '-c', '/home/will/data/xcp_testing/cohort.csv', '-r', '/home/will/data/xcp_testing/', '-i', '/home/will/data/xcp_testing/work', '-o', '/home/will/data/xcp_testing/output']

RUNNING: docker run --rm -it -v /home/will/data/xcp_testing/cbf.dsn:/design/design.dsn:ro -v /home/will/data/xcp_testing/cohort.csv:/cohort/cohort.csv:ro -v /home/will/data/xcp_testing/output:/xcpOutput -v /home/will/data/xcp_testing/:/relative -v /home/will/data/xcp_testing/work:/scratch pennbbl/xcpengine:latest -d /design/design.dsn -c /cohort/cohort.csv -o /xcpOutput -m s -r /relative -i /scratch
/xcpEngine/xcpEngine: line 12: activate: No such file or directory
[Detected that xcp is in a container]
Received options: -d /design/design.dsn -c /cohort/cohort.csv -o /xcpOutput -m s -r /relative -i /scratch
$XCPEDIR is /xcpEngine

Constructing a pipeline based on user specifications
····································································
· [D][/design/design.dsn]
sed: cannot rename /design/sedIERqpm: Device or resource busy
· [I][/scratch]
· [C][/cohort/cohort.csv]
Error in file(file, ifelse(append, "a", "w")) : 
  cannot open the connection
Calls: write.table -> file
In addition: Warning message:
In file(file, ifelse(append, "a", "w")) :
  cannot open file '/cohort/cohort.csv': Read-only file system
Execution halted
· [O][/xcpOutput]
· [M][Executing serially]

From group/error/cbf_root-201907091932-9001

· [A major error has occurred.]
· [The processing stream will now abort.]
· [Preparing diagnostics]
····································································



Module Workflow Map
····································································

· @1
· @1.1
· @1.1a
· @1.1b
· @1.2
· @1.3
· @1.4
· @1.5
· @1.5.1
· @1.5.2a
· @1.5.3a
· @1.5
· @1.6
· @2
· @2.1
· ERROR
····································································



····································································
· [An error occurred while processing module localiser.]
· [The error was detected at signpost @2.1.]
· [The most recent command logged was]
· []
· [For additional details, view the processing log. For improved]
· [diagnostics, increment verbosity using the -t option.]
· []
· [The primary analyte is not a valid image: /home/will/data/xcp_testing/SPIRAL_V20_HCP_852307506.nii.gz]
· []
· [stream abort]
····································································

Runtime Information
Ran with Docker

Additional context
I don't think the directory containing the prepackaged pipelines was created, as mentioned in Step 2 here.

To setup xcp, I pip installed the xcpengine-container package.

[a3sha2]

Hi @willtack

The error is from binding. How did you call it from command line?

[willtack]

This is was my command line call:

will@desk:~$ xcpengine-docker \
-d /home/will/data/xcp_testing/cbf.dsn   \
-c /home/will/data/xcp_testing/cohort.csv \
-r /home/will/data/xcp_testing/   \
-i /home/will/data/xcp_testing/work  \
-o /home/will/data/xcp_testing/output

[a3sha2]

the issue is in your cohort file. the img,m0 and anat must have the same relative ( -r)

[willtack]

Hmm, I tried adjusting my paths, with the same -r as before.

id0,img,m0,anatdir
sub-P68,SPIRAL_V20_HCP_852307506.nii.gz,SPIRAL_V20_HCP_M0_Eq_1.nii.gz,fmriprep/sub-P68/anat

For reference, this is how my directory is set up:

└── xcp_testing
    ├── cbf.dsn
    ├── cohort.csv
    ├── fmriprep
    ├── output
    ├── SPIRAL_V20_HCP_852307506.nii.gz
    ├── SPIRAL_V20_HCP_M0_Eq_1.nii.gz
    └── work

I also tried with absolute paths:

id0,img,m0,anatdir
sub-P68,/home/will/data/xcp_testing/SPIRAL_V20_HCP_852307506.nii.gz,/home/will/data/xcp_testing/SPIRAL_V20_HCP_M0_Eq_1.nii.gz,/home/will/data/xcp_testing/fmriprep/sub-P68/anat

With the same results as above.

[a3sha2]

pls can you use docker image or singularity directly?

[willtack]

I tried the docker image directly:

docker run --rm -it -v /data:/home/will/data \
pennbbl/xcpengine:latest \
 -c /home/will/data/xcp_testing/cohort.csv \
 -d /home/will/data/xcp_testing/cbf.dsn \
 -o /home/will/data/xcp_testing/output \
 -i /home/will/data/xcp_testing/work/ \

And received this error. Similar to above but it also can't seem to find the design file. The design file is definitely at that location and identical to what's posted in this repository:

docker run --rm -it -v /data:/home/will/data pennbbl/xcpengine:latest  -c /home/will/data/xcp_testing/cohort.csv -d /home/will/data/xcp_testing/cbf.dsn  -o /home/will/data/xcp_testing/output -i /home/will/data/xcp_testing/work/
/xcpEngine/xcpEngine: line 12: activate: No such file or directory
[Detected that xcp is in a container]
Received options: -c /home/will/data/xcp_testing/cohort.csv -d /home/will/data/xcp_testing/cbf.dsn -o /home/will/data/xcp_testing/output -i /home/will/data/xcp_testing/work/
$XCPEDIR is /xcpEngine

Constructing a pipeline based on user specifications
····································································
· [D]Missing or empty design file
· [D]Aborting
___________________________________________________________________

[Figsandcitrus]

Hello,

I am having the same issue where the output of the batch file that I am trying to submit gives me the same outcome as posted above that the design file is empty or not present. I cannot view the replies to @willtack's post from earlier. Any help is appreciated, thank you!

[a3sha2]

pls use the latest docker image

[Figsandcitrus]

I am running this on singularity as this is what we have been using at my lab. When running the script on its own, it works. However, when I have tried to submit as an sbatch, it is giving this message.

[a3sha2]

pull for new singularity image from dockerhub https://xcpengine.readthedocs.io/config/tutorial.html#before-you-begin