GSoC 2020 Final Code Submission

Project link — MRI Registration using Deep Learning and Implementation of Thin-Plate Splines
Organization — INCF and DIPY

Weekly Progress

• Community Bonding + Week 1

  • Developed a POC for non-rigid image registration using the MNIST dataset. Colab notebook
  • Made a pull request for the same. PR: dipy/dipy#2137
  • Cleaned the code and refashioned the dipy.nn.registration API
  • Added docstrings and comments.

• Week 2

  • Implemented the VoxelMorph model (CVPR 2018). But yet to be trained. Paper
  • Implemented and trained 2D and 3D deep learning models for unsupervised affine registration. Similar to AIRNet but trained in an unsupervised fashion like the VoxelMorph model.

• Week 3

  • Tested the VoxelMorph model on the sample data (150 T1-weighted images) that was provided. The results were very good.
  • Cleaned the code and tested the new dipy.nn.registration API that was refashioned in the first week. Colab notebook

• Week 4

  • Added tests and tutorial to the PR. dipy/dipy#2137

• Evaluation 1 Week

  • Tested the VoxelMorph model on a few T2-weighted volumes from the OASIS dataset.

• Week 5

  • Tried the VoxelMorph model for cross-modality registration (T1w static image and T2w moving images). Results weren't good.

• Week 6

  • Retrained the cross-modality registration model with better brain segmented T2 images. The brain mask was obtained using T1s insterad of T2s. So I rigid registered T1s with T2s and got the brain masks using T1s. Still the results were bad.
  • Implemented thin-plate splines. GitHub repo

• Week 7

  • Read and understood the N3 paper. (A Nonparametric Method for Automatic Correction of Intensity Nonuniformity in MRI Data)

• Week 8 + Evaluation 2 Week + Week 9

  • Tried the SyN algorithm for cross-modality registration. Results looked good but the ventricular regions were expanding.
  • Discussed doubts in N3 paper

• Week 10 and 11

  • Implemented N3 paper. But yet to be tested
  • Refashioned and cleaned the PR dipy/dipy#2137
  • Tried the SyN algorithm with more iterations. Results got worse.

• Week 12

  • Blogs and codes
  • Documenting the whole work

Blogs

• Deep-learning-based 2D Deformable Image Registration with MNIST
• Deep-learning-based 2D and 3D Affine Registration
• Deep-learning-based 3D Deformable Registration of T1-weighted images
• DIPY NN Registration API
• Deep-learning-based 3D Deformable Registration of T2-weighted images
• Deep-learning-based 3D Cross-modality Registration
• Thin-plate Splines

Codes

• Thin-plate splines - https://github.com/sarathknv/tps
• 3D Deformable Image Registration (VoxelMorph) - https://github.com/sarathknv/voxelmorph
• 2D Deformable Image Registration - dipy/dipy#2137
• N3 Bias Field Correction - https://colab.research.google.com/drive/1eHAkHgMPDK4tlxihgMfK73FnaiQpdAR4?usp=sharing

Colab Notebooks

• 2D Deformable Registration of MNIST Images
• 3D Deformable Registration of T1 Images
• 3D Deformable Registration of T2 Images
• 3D Deformable Cross-Modality Registration
• 2D Affine Registration of MNIST Images
• 3D Affine Registration of T1 Images
• Registration API Usage
• VoxelMorph (CVPR 2018)
• N3 Bias Field Correction