This repository is part of a project for a deep learning course, in which the RNSA Bone Age dataset is used, publicly available at Kaggle. The aim of the project was to experiment with fine-tuning of pre-trained networks, including both image and gender input for bone age assessment. The final deliverables of the project were a one-page paper with references squeezed unformatted in the footnote (paper.pdf) and source code.
Original images were resized to 256x256 pixels with padding. Code for resizing and padding can be found in the preprocessing directory. Additionally, real-time data augmentation on the training set is used during training. An example image can be found below.
| Original | Padded | Augmented |
|---|---|---|
 |
 |
 |
The model used in this project incorporates a fine-tuned pre-trained architecture (VGG16 or ResNet18) to handle the image input and a gender network. A schematic overview of the model can be found below.
The following architectures pre-trained on the ImageNet dataset were used: VGG16 and ResNet18. Next to the model architecture provided above, I trained two models (VGG16 and ResNet18) on image input only.
Training, tuning and evaluation procedures for the models including both gender and image inputs as shown schematically are presented in the following notebooks: notebooks/vgg16-concatenated.ipynb and resnet18-concatenated.ipynb and together form the majority part of this project.
Additionally, training and evaluation for the models including image input only can be found in the following notebooks: notebooks/vgg16.ipynb and notebooks/resnet18.ipynb. These models are trained with optimal hyperparameters found for the models including both gender and image inputs, instead of being separately tuned.
Note: to run the notebooks on the ResNet18 models, please install the following keras patch (pip install -U --force-reinstall --no-dependencies git+https://github.com/datumbox/keras@bugfix/trainable_bn) and tensorflow 1.14, due to issues on finetuning of pre-trained models including batch normalization layers. Read more here. The notebooks on the VGG16 models can be run on Keras 2.2.4 and tensorflow 2.2.
The table below presents the validation and test MAE in months for the models with optimal hyperparameter settings. Details on the optimal hyperparameter settings can be found in the accompanying notebooks in the notebooks directory. The best-performing model is the model incorporating the ResNet-18 architecture, including both image and gender inputs.
| Model | MAE (months) | |||
|---|---|---|---|---|
| Image + gender input | Image input only | |||
| Val | Test | Val | Test | |
| VGG16 (finetuned) | 10.6 | 15.9 | 12.1 | 16.9 |
| ResNet18 (finetuned) | 10.6 | 11.7 | 12.8 | 14.6 |