PET HRRT insertion pipeline

README.md

@@ -10,6 +10,7 @@ This repo can be installed on the same VM as the main LORIS codebase, or on a di
  * Python 3 with pip3 and virtualenv (step 2 below)
  * MINC toolkit (step 3 below)
  * DICOM toolkit (step 4 below)
+ * tpcclib (for HRRT PET only) (http://www.turkupetcentre.net/tpcclib-doc/md_install.html)
 
 On <u>Ubuntu</u>, DICOM toolkit will be installed by the imaging install script (step 4 below). This script will _apt-get install dcmtk_.   
 

batch_uploads_imageuploader.pl

@@ -161,15 +161,13 @@ =head2 Methods
 my ($stdoutbase, $stderrbase) = ("$data_dir/batch_output/imuploadstdout.log", 
 				 "$data_dir/batch_output/imuploadstderr.log");
 
-
-while($_ = $ARGV[0], /^-/) {
+while($_ = $ARGV[0] // '', /^-/) {
     shift;
     last if /^--$/; ## -- ends argument processing
     if (/^-D/) { $debug++ } ## debug level
     if (/^-v/) { $verbose++ } ## verbosity
 }
 
-
 ## read input from STDIN, store into array @inputs (`find ....... | this_script`)
 my @patientnamearray = ();
 my @fullpatharray = ();

dicom-archive/profileTemplate.pl

@@ -138,3 +138,51 @@ sub isFileToBeRegisteredGivenProtocol {
         return  undef;
     }
 }
+
+
+=pod
+
+=head3 determineHRRTprotocol($tracer, $ecat_file)
+
+Determines which protocol name to use for the ECAT file (based on the tracer
+name, reconstruction algorithm and motion correction used). For the MNI PET
+HRRT scanner, the following convention is being used:
+- the tracer used is passed to the function as $protocol (which comes from what
+was written in the Matlab script accompanying the dataset)
+- reconstruction is indicated in the filename by the string DFBP (for FBP
+reconstruction) and OSEM (for OSEM reconstruction)
+- whether motion correction was applied on the file is indicated by the string _MC01
+
+Note: this will only apply for the HRRT PET scanner from the MNI. Since there is
+only 7 of those scanners in the world, we will adapt the scripts if needed when
+we will start hosting data from other PET HRRT scanners.
+
+INPUTS:
+  - $tracer   : name of the tracer (that was previously parsed from the Matlab
+                file accompanying the PET HRRT dataset for the MNI)
+  - $ecat_file: name of the ECAT file
+
+RETURNS: the final acquisition protocol to be used when inserting the acquisition
+ in the files table.
+
+=cut
+
+sub determineHRRTprotocol {
+    my ( $tracer, $ecat_file ) = @_;
+
+    my $protocol = $tracer;
+    $protocol =~ s/'|\\|_|"|\s//g; # remove weird characters from protocol name
+
+    if ( $ecat_file =~ /Dfbp/i ) {
+        $protocol .= "FBP"; # add FBP reconstruction to filename
+    } else {
+        $protocol .= "OSEM"; # add OSEM reconstruction to filename
+    }
+
+    if ( $ecat_file =~ /_MC01/i ) {
+        $protocol .= "mc"; # add mc acronym for motion corrected datasets
+    }
+
+    return $protocol;
+}
+

docs/01-Introduction.md

@@ -23,16 +23,23 @@ and links them with corresponding behavioral data in LORIS. Scans can be viewed
 and quality controlled in the LORIS front end via web browser, facilitating 
 collaboration between radiologists, clinicians and researchers.
 
+In addition to the import of DICOM files, it is possible to upload a compressed 
+file containing an HRRT PET dataset. 
+Note that the HRRT insertion pipeline has been 
+developed using datasets produced at the Brain Imaging Center of the Montreal
+Neurological Institute. Some modification/customization might be needed for other
+HRRT PET scanners as datasets coming from those scanners are not standardized.
+
 ### 1.2 How does LORIS-MRI work? 
 
 ![user_story](images/user_story.png)
 
 LORIS-MRI allows multiple ways to upload scans, but typically, users
-upload a compressed (.tgz, .tar.gz, or .zip) DICOM folder via the Imaging
-Uploader module that should be composed of **only** DICOM files. LORIS requires 
-that the uploaded file name follow the naming convention 
+upload a compressed (.tgz, .tar.gz, or .zip) DICOM (or HRRT) folder via the Imaging
+Uploader module that should be composed of **only** DICOM (or HRRT) files. LORIS 
+requires that the uploaded file name follow the naming convention 
 `PSCID_CANDID_VISIT-LABEL`. 
-In addition, all DICOM datasets uploaded via the Imaging Uploader or 
+In addition, all DICOM/HRRT datasets uploaded via the Imaging Uploader or 
 transferred on the LORIS-MRI server must be free of any identifying 
 information (*e.g.* patient name). A tool can be provided to the sites to 
 facilitate de-identification. Please contact the LORIS team for details.
@@ -48,8 +55,8 @@ Imaging Uploader module, where descriptive messages can be consulted.
 The output of the main key steps in the insertion progress can also be consulted 
 through:
  - the LORIS DICOM Archive module for successfully archived DICOM datasets`*` 
- - the Imaging Browser module for MINC files (generated from DICOM) that pass the 
-study-defined MRI protocol`*`
+ - the Imaging Browser module for MINC files (generated from DICOM or HRRT ECAT7 
+files) that pass the study-defined MRI protocol`*`
  - BrainBrowser using 3D or 4D navigation of these MINC files. More details on 
 BrainBrowser's capabilities can be found [here.][5]
 

docs/02-Install.md

@@ -266,6 +266,11 @@ if($acquisitionProtocol eq 't1' or $acquisitionProtocol eq 't2' or $acquisitionP
 - `get_DTI_CandID_Visit()`
 
     Used by the DTIPrep pipeline
+
+- `determineHRRTprotocol()`
+
+    Routine to determine the acquisition protocol to use to register an HRRT derived
+    file.
 
 
 ## <a name="post-installation-checks"> 2.3 Post-installation checks
@@ -385,7 +390,30 @@ The graph below shows the different modules mentioned above with the
 ![pipeline_flow](images/overall_flow.png)
 
 
-### 2.4.2 BIDS insertion (Electrophysiology and Imaging)
+### 2.4.2 PET data from an HRRT scanner
+
+The pipeline was initially designed for **raw HRRT PET datasets collected at 
+the Brain Imaging Center of the Montreal Neurological Institute**. Since there is
+no standard for HRRT datasets and only 7 scanners existing in the world, the
+insertion pipeline of PET data from an HRRT scanner might need to be updated/modified
+for other scanners and will be done when the need comes.
+
+Image insertion of PET dataset from an HRRT scanner is very similar to the 
+insertion described for DICOM datasets, to the exception that the HRRT archive
+information are stored in the `hrrt_archive` tables instead of the `tarchive` tables.
+
+Insertion into LORIS is performed via the following steps:
+
+1. Upload of a compressed set of the HRRT PET study via the **Imaging Uploader**
+     module (or transfer the compressed HRRT PET studies directly on the server
+     hosting LORIS).
+2. HRRT insertion into the `hrrt_archive` tables.
+3. Conversion of the ECAT7 files into MINC files for protocol identification and
+     insertion into the LORIS database. Note that the ECAT7 images will be linked
+     to the inserted MINC files. 
+
+
+### 2.4.3 BIDS insertion (Electrophysiology and Imaging)
 
 The pipeline to insert BIDS datasets into LORIS currently support the 
 following BIDS modalities/entities:

docs/03-TechnicalInfrastructure.md

@@ -18,6 +18,7 @@ The root directory of the imaging part of a LORIS instance is typically
             |__ assembly
             |__ bids_imports
             |__ batch_output
+            |__ hrrtarchive
             |__ logs
                 |__ DTIPrep_pipeline*
                 |__ DTIPrep_register*
@@ -98,6 +99,25 @@ For the example mentioned here, the sub-directory would be named:
 Within that BIDS sub-directory, the file system structure will follow the 
 [BIDS specification][1].
 
+
+#### The `hrrtarchive` directory
+
+The HRRT archives listed in the `hrrt_archive` table are stored in the
+  `data/hrrtarchive` directory and organized within folders representing the 
+  different years of acquisition.
+
+```
+## Content of the /data/$PROJECT/data/hrrttarchive directory
+.
+|__ year_1
+    |__ DCM_`date`_hrrtarchive.tar
+    |__ DCM_`date`_hrrtarchive.tar
+    |__ DCM_`date`_hrrtarchive.tar
+|__ year_2
+    |__ DCM_`date`_hrrtarchive.tar
+    |__ DCM_`date`_hrrtarchive.tar
+    |__ DCM_`date`_hrrtarchive.tar
+```
 
 
 #### The `logs` directory
@@ -252,10 +272,28 @@ In the front end of LORIS, you can see the DICOM studies using the
 
 Note: the `SessionID` field of the `tarchive` table is populated once at least  
   one MINC file derived from that DICOM study got inserted in the tables 
-  described in 3.2.3.
+  described in 3.2.1.3.
+
 
+#### 3.2.1.2 HRRT archive tables
 
-#### 3.2.1.2 Files tables
+The first step to insert a new HRRT session into the database is the 
+  insertion of the HRRT PET study. In the database, all information related to a
+  HRRT PET study is being organized into two different tables:
+
+ * the `hrrt_archive` table stores information about the whole HRRT session,
+     including patient, center name and study information, as well as the location 
+     of the archived HRRT dataset. Each row correspond to a specific HRRT 
+     session.
+ * the `hrrt_archive_files` table stores information about each ECAT7 file found in 
+     the HRRT session. Each row correspond to one ECAT7 file and is linked 
+     to the `hrrt_archive` table via the `HrrtArchiveID` foreign key.
+
+Note: the `SessionID` field of the `hrrt_archive` table is populated once at least  
+  one MINC file derived from that HRRT study got inserted in the tables 
+  described in 3.2.1.3.
+
+#### 3.2.1.3 Files tables
 
 The second step to insert a new imaging session into the database is the 
   conversion of the DICOM study into the MINC files that will be inserted based 

docs/04-Scripts.md

@@ -112,7 +112,7 @@ Note carefully the following arguments:
 
 - `-acquisition_protocol`: must be a known scan type according to the
     `mri_scan_type` table
-- `-tarchive_Path`: the DICOM tarball
+- `-tarchive_Path`: the DICOM or HRRT tarball
 - `-mincPath`: note this file may have been placed in the `trashbin` directory
 
 See also: [MRI-PR#141](https://github.com/aces/Loris-MRI/pull/141) for more

docs/05-PipelineLaunchOptions.md

@@ -1,6 +1,6 @@
 # 5.0 - Pipeline Launch options
 
-## 5.1 - Pipeline Launch Options for DICOM datasets
+## 5.1 - Pipeline Launch Options for DICOM and HRRT datasets
 
 Scans upload into LORIS and insertion pipeline's triggering can be done in a few 
 ways depending on each project's needs. The best choice will depend on the study 

imaging_install.sh

@@ -79,6 +79,9 @@ sudo -S cpan install Moose
 sudo -S cpan install MooseX::Privacy
 sudo -S cpan install TryCatch
 sudo -S cpan install Throwable
+sudo -S cpan install Digest::BLAKE2
+sudo -S cpan install File::Type
+sudo -S cpan install String::ShellQuote
 echo
 
 ################################################################################
@@ -111,6 +114,7 @@ echo "Creating the data directories"
   sudo -S su $USER -c "mkdir -m 2770 -p /data/$PROJ/data/"
   sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/trashbin"         #holds mincs that didn't match protocol
   sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/tarchive"         #holds tared dicom-folder
+  sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/hrrtarchive"      #holds tared hrrt-folder
   sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/pic"              #holds jpegs generated for the MRI-browser
   sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/logs"             #holds logs from pipeline script
   sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/assembly"         #holds the MINC files

imaging_install_MacOSX.sh

@@ -45,6 +45,7 @@ echo "Creating the data directories"
   sudo -S su $USER -c "chmod g+s /data/$PROJ/data/"
   sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/trashbin"          #holds mincs that didn't match protocol
   sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/tarchive"          #holds tared dicom-folder
+  sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/hrrtarchive"       #holds tared hrrt-folder
   sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/pic"               #holds jpegs generated for the MRI-browser
   sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/logs"              #holds logs from pipeline script
   sudo -S su $USER -c "mkdir -m 770 -p /data/$PROJ/data/assembly"          #holds the MINC files