Skip to content

[TMI'18] Workflow Recognition from Surgical Videos using Recurrent Convolutional Network, winner algorithm at MICCAI'16 M2CAI challenge

License

Notifications You must be signed in to change notification settings

carrenD/SV-RCNet

 
 

Repository files navigation

SV-RCNet

SV-RCNet: Workflow Recognition from Surgical Videos using Recurrent Convolutional Network (TMI 2018)

by Yueming Jin, Qi Dou, Hao Chen, Lequan Yu, Chi-Wing Fu, Jing Qin and Pheng Ann Heng

Introduction

The SV-RCNet repository contains the codes used in 2016 M2CAI workflow challenge and our SV-RCNet paper. Our method ranks the first in the M2CAI challenge and achieves a promising performance in one large surgical dataset, i.e., Cholec80 dataset.

The implementation is based on Ubuntu 14.04, CUDA 8.0, cuDNN 5.0, Anaconda 2.7 and Caffe.

Installation

  1. Clone the SV-RCNet repository
    git clone https://github.com/YuemingJin/SV-RCNet.git
  2. Build
    cd SV-RCNet
    cp Makefile.config.example Makefile.config
    # Adjust Makefile.config
    # Or directly use provided 'Makefile.config' file in the folder, which we have adjusted the necessary configurations, such as setting "WITH_PYTHON_LAYER := 1".
    make all -j8
    make pycaffe

Note:

  • Please first install Anaconda 2.7 following official instructions. In addition, adjust path in your 'Makefile.config' file.
  • For other installation issues, please follow the official instructions of Caffe.

Step by Step Recognition

Most related codes are in surgicalVideo/ folder.

  1. Download data

    Cholec80 dataset or M2CAI dataset

  2. Preprocess data

  • Download ffmpeg and use ffmpeg to split the videos to image. We split the videos in 1 fps for Cholec80 and only split video01 as an example.

    cd surgicalVideo
    sh split_video_to_image.sh 
  • Resize the image from 1920 x 1080 to 250 x 250.

Note: may need to modify the ground truth file (gt_file_Cholec80) according to the name of images you created.

  1. Training the network
  • Download pre-trained ResNet-50 model at https://github.com/KaimingHe/deep-residual-networks. Put it in models/ResNet-50/.
  • Enter models/ResNet-50 and modify path in ResNet-50-workflow-train-val.prototxt and pre-trained model name in train_ResNet_50.sh.
  • Train ResNet-50
    sh train_ResNet_50.sh 
  • The trained ResNet-50 will be saved in snapshot/ folder. Please choose and copy the model to the models/SV-RCNet/ folder as the next step pre-trained model when the loss does not decrease.
  • Enter python/ folder and modify paths in set_input_layer.py
  • Enter models/SV-RCNet and modify pre-trained model name in train_SVRCNet.sh.
  • Train SV-RCNet
    sh train_SVRCNet.sh
  1. Testing

    Enter test/ folder to inference all the testing videos. Need to change paths in test.py.

    python test.py

Citation

If you make use of the code, please cite the paper in resulting publications.

@article{jin2018sv,
  title={SV-RCNet: Workflow Recognition From Surgical Videos Using Recurrent Convolutional Network}, <br />
  author={Jin, Yueming and Dou, Qi and Chen, Hao and Yu, Lequan and Qin, Jing and Fu, Chi-Wing and Heng, Pheng Ann}, 
  journal={IEEE transactions on medical imaging},
  volume={37},
  number={5},
  pages={1114--1126}, 
  year={2018},
  publisher={IEEE}
}

Contact

Any query, please email at ymjin@cse.cuhk.edu.hk and qdou@cse.cuhk.edu.hk

About

[TMI'18] Workflow Recognition from Surgical Videos using Recurrent Convolutional Network, winner algorithm at MICCAI'16 M2CAI challenge

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 79.5%
  • Python 9.5%
  • Cuda 5.6%
  • CMake 2.8%
  • MATLAB 1.3%
  • Makefile 0.7%
  • Other 0.6%