RNFLT2Vec

The code for the paper entitled RNFLT2Vec: Artifact-Corrected Representation Learning for Retinal Nerve Fiber Layer Thickness Maps published in the Medical Image Analysis. If you have any questions, please email harvardophai@gmail.com and harvardairobotics@gmail.com.

Requirements

Python 3.8
tensorflow 2.4.0
opencv-python 4.5.5

Dataset

Here are sample codes to visualize the RNFLT map:

from utils.map_handler import *
import matplotlib.pyplot as plt

rnflts = np.load('dataset/samples.npy')
img = rnflts[0]
plot_2dmap(img)

Calculate the masked RNFLT and corresponding mask image:

# RNTLT values less than the threshold are treated as artifacts
masked_map, ori_mask, resized_map = process(img, threshold=50)

plt.imshow(masked_map)
plt.show()
plt.imshow(ori_mask)

Pretrained Model

The model weight "combined_rnflt2vec_weights_512_128_10_0001_004.93-0.03" trained using 10,000 samples from our larger private dataset can be downloaded via this link

Use the Model

from models import rnflt2vec

# load the pretrained model
rnflt2vec = rnflt2vec.construct_model_from_args(args)
rnflt2vec.load('combined_rnflt2vec_weights_512_128_10_0001_004.93-0.03', train_bn=False, lr=0.00005)

# embedding learning model
encoder = rnflt2vec.model.get_layer('embed_model')
model_embed = Model(inputs=encoder.inputs, 
                    outputs=encoder.get_layer('encoder_output').output)
                    
# artifact correction model                   
rnflt2vec_inpaint_model = rnflt2vec.model.get_layer('inpaint_model')
model_correction = Model(inputs=RNFLT2Vec_inpaint_model.inputs, outputs=RNFLT2Vec_inpaint_model.outputs)
                                 
# embedding inference
embeds = model_embed.predict([masked_map, ori_mask])[0] 
# artifact correction
pred = model_correction.predict([masked_map, ori_mask])[0]
plot_2dmap(pred, show_cup=True)

Artifact correction example:

Acknowledgement and Citation

If you find this repository useful for your research, please consider citing our paper:

@article{shi2024rnflt2vec,
  title={RNFLT2Vec: Artifact-corrected representation learning for retinal nerve fiber layer thickness maps},
  author={Shi, Min and Tian, Yu and Luo, Yan and Elze, Tobias and Wang, Mengyu},
  journal={Medical Image Analysis},
  pages={103110},
  year={2024},
  publisher={Elsevier}
}