Training

[MarijnJABoer]

Tuning hyperparameters

• Steps per epoch (samples//batch_size) (the same as iterations per epoch? - Yes!?)
• Optimizer (nadam!?)
  • Tune the optimizers
• Epoch (related to how diverse your data is)
• Batch size (because of the low quality images, we can increase this?) link
• Dropout (what percentage?)
• Implement hyperparameter grid search?

Training

• Load model from build.py Models (CNN's) #8
• Load model from models.py Models (CNN's) #8
• Earlystopping
• ReduceLROnPlateau
• Move ImageDataGenerator from data_input.py to train.py
• Split train_generator and train_datagen in train.py
  • Implement this fix in show_augmentation()
• Multi-GPU model link

On model saving with Multi-GPU
To save the multi-gpu model, use .save(fname) or .save_weights(fname) with the template model (the argument you passed to multi_gpu_model), rather than the model returned by multi_gpu_model.

• Load model using imagenet weights
• Metrics to csv
• Add Inception-ResNet v2
• Building my own image data generator link
  • .flow_from_directory() is faster than .flow()
  • link
• Import callbacks correctly
• Pass training to object (history = model.fit())

model.fit() returns history object which contains information about training an validation accuracy and loss:

{'acc': [0.9843952109499714],
 'loss': [0.050826362343496051],
 'val_acc': [0.98403786838658314],
 'val_loss': [0.0502210383056177]
}

• plot metrics using python (with ggplot theme)
• is categorical_crossentropy the correct loss function?

Metrics

• Fix metrics=['accuracy']
• Add RMSE
• Add categorical_accuracy
• Add more metrics? link
  • MAE
  • MSE
• Add classification metrics for top 3 and top 5 top_k_categorical_accuracy

Others

• Is resizing / augmentation working?
• standardization vs normalization vs generalization vs regularization?
• Choose a verbose
  • verbose: 0, 1, or 2. Verbosity mode. 0 = silent, 1 = progress bar, 2 = one line per epoch.
• Sessions

    sess = tf.session()
    K.set_session(sess)