Deploying AI models

Pre-trained models

Windows and Mac OS

AI models are distributed as packaged Conda environments, available upon request through MSKCC Box.

To untar the environment file, for example in Windows, open the Command Prompt and run the following with correct paths of the tar file and the destination directory: tar -xf CT_heart_sub_structs.tar.gz -C CT_heart_sub_structs

Linux-based OS

Models are distributed as Conda environments as well as Singularity containers.

---

Requirements

• MATLAB or GNU-Octave
  CERR's AI model deployment pipeline is compatible with Matlab as well as Octave for license-free use.

• CERR
  Download CERR and add it to your MATLAB/Octave path:

addpath(genpath('/your/path/to/CERR/'));

• CUDA
  AI models can be executed in CPU-only or GPU-mode. For the latter, an NVIDIA GPU and the associated CUDA-capable driver are required.

• AI model
  AI models along with dependencies are available in ready-to-use Singularity containers (Linux) or Conda archives (Windows, Mac, Linux) hosted at MSKCC. Please submit a request for access to pre-trained models.

  If using Singularity containers, download Singularity 3.5 or higher on your Linux machine. For Windows OS we provide instructions to set-up Singularity over Vagrant.

  See documentation on currently available pre-trained models.

Usage

Editing the configuration file

CERR uses JSON-format configuration files to specify pre/post processing transformations to models inputs/outputs). Configuration files for pretrained models are distributed with CERR.

Configurations may need to be tailored to your dataset, e.g. if preprocessing involves cropping around a structure prior, the structure name may need to be modified to match the labeling conventions of your dataset. This is done by modifying the structureName value under the crop field in the configuration file.

Further details of supported configurations are described here.

Accessing planC

• From GUI

Load the CERR file in the Viewer and run:

global planC

• From file

lungExampleCerrFile = fullfile(getCERRPath,'..','Unit_Testing',...
    'data_for_cerr_tests','CERR_plans','lung_ex1_20may03.mat.bz2');
planC = loadPlanC(lungExampleCerrFile,tempdir);
planC = updatePlanFields(planC);
planC = quality_assure_planC(baseCerrFile,planC);

Executing the container

The following command can be used to deploy AI models in most cases:

With DICOM-format input

input_dicom_dir:Directory containing DICOM images and associated RT objects for all the modalities required by the model. We recommend creating a new writable directory to house these results.

output_dicom_dir is the path to which outputs (e.g. DVFs, segmentations as DICOM RTSTRUCTs, CERR archives) are written.

save_flag is set to 'Yes' to save CERR-format (planC) archives in addition to DICOM (or 'No' to skip).

With planC-format input

planC: CERR-specific data structure.

• session_dir Running the segmentation container creates a temporary session directory which is deleted once execution ends. The user must provide a "writable" location to create this temporary directory. We suggest making a new directory on the server to house temporary sessions.

• algorithm is the name of the segmentation algorithm to execute (see list of valid algorithm names above). To run multiple algorithms in sequence, separate the names using the "^" sign, e.g. 'CT_Heart_DeepLab^CT_Atria_DeepLab'

• cmd_flag is set to singContainer (Singularity Containers) or condaEnv (Conda Environemnts).

• container_path is the full path to the supplied Singularity container.

• new_session flag is set to '0' to use existing session directory (e.g. for sequential application of models) or '1' to create a new directory.

• waitbar_handle is a handle to a progress bar

• ssh_config This functionality is currently untested and we recommend passing an empty ([]) value. This flag is intended to support invoking the model on a remote (Linux) server.

The following inputs are optional input:

• inputS is a data structure containing scan indices input to the model (inferred from scan identifier field of configuration file by default).
• output_scan_no is the index of the scan to be associated with model output. This field is relevant for multi-input models. It can be instead inferred from the outputAssocScan field of configuration file if provided.
• skip_mask_export flag is set to '0' if structure masks must be input to the model (turned off ('1') by default).

Sample call

% Initialize dcm4che to create temp dir names
init_ML_DICOM

% Define I/O paths
inDcm = '/path/to/inputDcmDir';
outDcm = '/path/to/outputDcmDir';
sessionPath = '/path/to/segSessions';
container_path = '/path/to/CT_Heart_DeepLab_sing_conda_recipe.def_363df.sif'; % e.g.for cardiac substructures

% Specify algorithm<sup>a</sup>
algorithm = 'CT_HeartStructure_DeepLab^CT_HeartSubStructures_DeepLab^CT_Atria_DeepLab^CT_Pericardium_DeepLab^CT_Ventricles_DeepLab';
% algorithm = 'CT_LungOAR_incrMRRN'; % For Thoracic OAR segmentation model

%Specify scan to segment 
scanNumToSegment = 1;
inputS.scan.scanNum = scanNumToSegment;   % User-specifies 1st scan is to be segmented  
outputAssocScan = scanNumToSegment;       % User-specifies 1st scan is to be associated with result

% Set optional flags
ssh_config= '';
waitbar_handle= nan;
cmd_flag= 'singContainer';
new_session= true;
skip_mask_export = true;                  %Set to false if model requires input masks

• ^a See documentation for supported options ("Apps in Container").

DICOM-based

runAIforDicom(input_dicom_dir,output_dicom_dir,session_dir,algorithm,cmd_flag,save_flag,container_path,inputS,outputAssocScan,skip_mask_export );

planC-based

planC = runAIforPlanC(planC,session_dir,algorithm,cmd_flag,new_session,ssh_config,waitbar_handle,container_path,inputS,outputAssocScan,skip_mask_export);

Demo AI model deployment via CERR

This video demonstrates the deployment of an AI model to segment cardiac substructures in CT using a packaged conda environment

---

Citation

1. See documentation for references to pretrained models.
2. Iyer, A., Locastro, E., Apte, A. P., Veeraraghavan, H., & Deasy, J. O. (2021). Portable framework to deploy deep learning segmentation models for medical images. bioRxiv. https://doi.org/10.1101/2021.03.17.435903
3. Apte, A. P., Iyer, A., Thor, M., Pandya, R., Haq, R., Jiang, J., LoCastro, E., Shukla-Dave, A., Sasankan, N., Xiao, Y., Hu, Y. C., Elguindi, S., Veeraraghavan, H., Oh, J. H., Jackson, A., & Deasy, J. O. (2020). Library of deep-learning image segmentation and outcomes model-implementations. Physica Medica, 73, 190-196. https://doi.org/10.1016/j.ejmp.2020.04.011.