question about selecting best model (validation or post last step?)

[shjustinbaek]

Hi, I have been reading your paper and code,
and I am confused about how the best model of the entire training process is selected.

this is how I understood the training code

1. model training (both burn-in and mutual learning stage) is performed on train data
2. model weight is saved every 5000 steps, by hooks.PeriodicCheckpointer
3. After the last training step is finished (MAX_ITER reached), resulting weight is used for evaluation

Please correct me if i am wrong.

and my questions are:
a. Should I take the model weight after the last training step as the final model weight for future inference?
b. It seems validation loss/metric is not calculated in the code, but in the paper there is a plot of validation mAP (Figure 4 )
Are the metrics reported on the paper calculated with post last training step weights or weight selected based on validation set?
c. Is there a model selection based on validation loss/metric function that i missed in this repo?

Thank you for the great paper and code
I found the contents really interesting.
Thanks in advance!

[yujheli]

a. I usually test the model weight of the middle training scenario which has better performance.
b. I calculate them in tensorboard. I downloaded the csv file and draw the curve using python matplotlib.
c. I select the best model from the tensorboard curve and go back to search for the saved checkpoint.