Image Style Transfer for Brain Vessel Segmentation from multi-modal MRI - Tensorflow v2

EURECOM SEMESTER PROJECT - SPRING 2022

Supervisors: Professor M. Zuluaga - F. Galati

Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation

Tensorflow v2 implementation of the SIFA unsupervised cross-modality domain adaptation framework.
Please refer to the branch SIFA-v1 for the original paper and code

Original Paper

Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation
IEEE Transactions on Medical Imaging

How to run

  !git clone https://github.com/2Falx/SIFA-Tensorflow2.git

  !mv {your_data_folder} SIFA-Tensorflow2

  %cd SIFA-Tensorflow2/

• Open and follow the "SIFA_Implementation_Tf2-.ipynb" jupyter notebook modifying the indicated variable and parameters to adapt it to your dataset.

• Set the home_path in Cell #3 (here it corresponds to the same folder of the above mentioned notebook).

• Store your data inside a folder "data" in the home_path

• Store SWI images in /data/SWI and TOF images in /data/TOF (You can find an example folder for a single 3D image in cell #6)

• Select the desired Spacing in cell #13 to allow images resizing (here SWI Spacing is chosen)

• Cell #23 contains all the useful statistics for OUR dataset: comment the last line to use the extracted ones from your data

• Slices are preprocessed in Cell #27 and transformed into Tfrecords in Cell #30 (Here you can modify the "tfrecords_folder" name)

• Modify the "percentage" parameter in the "split_file" function (Cell #31) to select the desired train/validation split percentage

• You can use (or modify) the function in Cell #34 to remove previous SIFA Training outputs

• Cell #37 contains a brief description of all the parameters you can set before the training

• Start Training by running Cell #38

• "Print losses" Section contains some Cells in order to create a "tot_losses.csv" file containing the loss-trend for each model loss given the name of an output folder ("date" variable, Cell #41). You can find your file inside the "{date}/losses" path.

• You can also uncomment the last lines of Cell #42 to remove all the saved model of the selected output to save some space

• Last Section contains some Cells in order to plot all the Loss-Curves given the complete path of a .csv file ("file_name " variable, Cell #46). You can find the list of all the .csv in your home path by running Cell #43 or #45

Data Preparation

• Spacing Extraction
• Slicing and Reshaping
• Standardization using mean and std over the Volume of the slice
• Normalization using min and max over the whole dataset
• Slice Preprocessing:
  1. Center Crop
  2. Zero Padding
  3. Add 1 dim / One-hot-Encoding
  4. Transformation into Tfrecord

Training Result

• You can find all the output results in the folder "{home_path}/SIFA/ouput/{date}"
• imgs: Contains all the .jpg images that you can also find in the .html files
• nib_imgs: Same of imgs but contains all the .nii.gz
• losses: Contains the .txt files of all the loss, both individually in each file and all-toghether in tot_losses.csv

html visualization files:

• 1st Row: 6 Data images

  1. $X_s$ : Input from the Source Domain
  2. $X_t$ : Input from the Target Domain
  3. $X_{s &#8594 t}$ : Transformed Image from Source to Target
  4. $X_{t &#8594 s}$ : Transformed Image from Target to Source
  5. $X_{s~}$ : Reconstructed Source Image ($X_{s &#8594 t &#8594 s}$)
  6. $X_{t~}$ : Reconstructed Source Image ($X_{t &#8594 s &#8594 t}$)

• 2nd Row: 7 Segmentation masks

  1. 'pred_mask_a' : Predicted Mask from $X_t$ ($Y_t$)
  2. 'pred_mask_b' : Same as 'pred_mask_a' ¹
  3. 'pred_mask_b_ll' : Predicted Mask from $X_{t &#8594 s}$ ($Y_{t &#8594 s}$)
  4. 'pred_mask_fake_a' : Predicted Mask from $X_s$ ($Y_s$)
  5. 'pred_mask_fake_b' : Same as 'pred_mask_fake_a' ¹
  6. 'pred_mask_fake_b_ll' : Predicted Mask from $X_{s &#8594 t}$ ($Y_{s &#8594 t}$)
  7. 'gt' : Ground truth, $X_s$ given Mask ($Y_s$)

Acknowledgement

The code is a revisiting version of the original SIFA Implementation Part of the code is revised from the Tensorflow implementation of CycleGAN.

Notes

1: Check model.py get_outputs() returns for further details