transfer learning pathway in zoo

[dbuscombe-usgs]

No description provided.

[dbuscombe-usgs]

Sometimes, model weights and config files are available for a generic task, such as finding water

Transfer learning is a generic term that could encompass at least the following two scenarios

1. using the 'HOT_START' in the config file. Provide existing model weights to a new 'train_model.py' session in Gym, but train on a new dataset with the same classes. For example, I have a model for water/no water rained using satellite imagery, and I want to 'hot start' training for a similar model for aerial imagery

2. perhaps more classically, it could involve freezing encoder/decoder layers on the unet that has already been trained, and then further training with a new dataset

[ebgoldstein]

The goal for me is to understand if we would need fewer training examples to make a model using trnasfer learning..
so imagine we had 200 labeled images:
use 50 for training/validation
hold 50 for testing
and keep 100 out of the pictuere for now.

with an existing model:

1. replace last convolution layer with one of same x,y size, but different channels (num_classes)
2. freeze all layers except this last one, and run model for a few epochs with normal learning rate and the 50 train/val images
3. unfreeze entire model and train with low learning rate for a long time with the 50 train./val images
4. test against the 50 hold out images.

Now start at step 1 with 100 images for train'val... then again with 150 images for train/val... and consistently test on the 50 hold-out test set... Does improvement in model performance keep going up w/ more examples? or does it saturate?

[dbuscombe-usgs]

This would be a good experiment. It would be a third scenario...

3. freeze model layers, and also modifying classifying layer to accommodate a different number of target classes

[dbuscombe-usgs]

@ebgoldstein now we have a segformer model that allows for fine-tuning, so we are one step closer to this?