ModeTv2: GPU-accelerated Motion Decomposition Transformer for Pairwise Optimization in Medical Image Registration

By Haiqiao Wang, Zhuoyuan Wang, Dong Ni, Yi Wang.

Paper link: [arxiv]

News

(28/12/2024) We implemented the CUDA version of the 3D Correlation layer based on the modet package. For specific usage, see Correlation&modet.md

ModeTv1(ModeT) links： [paper] [code]

Dataset

The official access addresses of the public data sets are as follows：

LPBA [link]

Mindboggle [link]

ABCT [link]

Instructions and System Environment

The way of installation of modet package:

cd modet

pip install .

Please note that the modet package requires CUDA to be installed in the system's environment rather than the cudatoolkit package from the conda environment. If the cudatoolkit package is already present in the environment, please ensure it matches the system's CUDA version.

Our successfully installed environment for modet is as follows:

• Ubuntu 22.04
• pip 21.2.4
• gcc 9.5.0
• CUDA 11.3/11.8/12.1
• Python 3.9/3.11/3.12
• PyTorch 1.11.0/2.4.1
• NumPy 1.21.5
• Nvidia Tesla V100/Nvidia RTX 2080Ti/Nvidia RTX 3090

For convenience, we are sharing the preprocessed LPBA dataset used in our experiments. Once uncompressed, simply modify the "LPBA_path" in train.py to the path name of the extracted data. Next, you can execute train.py to train the network, and after training, you can run infer.py to test the network performance. The small version of ModeTv2 and ModeTv2-diff can run on 2080ti on our preprocessed LPBA dataset. Please note that the suffix "_diff" denotes the diffeomorphic model.

Citation

If you find the code useful, please cite our paper.

@misc{wang2024modetv2,
      title={ModeTv2: GPU-accelerated Motion Decomposition Transformer for Pairwise Optimization in Medical Image Registration}, 
      author={Haiqiao Wang and Zhuoyuan Wang and Dong Ni and Yi Wang},
      year={2024},
      eprint={2403.16526},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

The overall framework and some network components of the code are heavily based on TransMorph and VoxelMorph. We are very grateful for their contributions. The file makePklDataset.py shows how to make a pkl dataset from the original LPBA dataset. If you have any other questions about the .pkl format, please refer to the github page of [TransMorph_on_IXI].

Baseline Methods

Several PyTorch implementations of some baseline methods can be found at [SmileCode].

How can other datasets be used in this code?

This is a common question, and please refer to the github page of ChangeDataset.md for more information.