TRAINS Integration with keras-tuner

[bmartinn]

Hi Guys,

I love what you guys are building here, really nice and clean interface to use :)

I thought it might be beneficial to add a deeper integration between TRAINS and keras-tuner.
Specifically to add a TrainsTunerLogger to be used with keras.tuner so that users will have a summary of all the different runs.

At the moment if you add a TensorBoard callback and initialize Trains, all the experiments (and graphs) will be logged,
but what seems to me may add additional value, would be a summary of everything.

This is what I came up with, what do you think?

class TrainsTunerLogger(Logger):
    ... 

tuner = kt.Hyperband(
    build_model,
    project_name='kt examples',
    logger=TrainsTunerLogger(),
    objective='val_accuracy',
    max_epochs=30,
    hyperband_iterations=2)

[haifeng-jin]

@bmartinn Thank you for the suggestion. My questions is: Are we able to achieve this without the pull request? I think you can achieve this by passing the logger to tuner, right?

[bmartinn]

@haifeng-jin Yes you are correct :)
My thinking is that it's best to have the TrainsTunerLogger as part of the keras-tuner as it is is specific to kera-tuner (and loggers). That said it could also be part of Trains, and then we could have an example/documentation in keras-tuner like cifar10 with the training summary and maybe a few screenshots ...
What do you think?

[haifeng-jin]

@bmartinn I think it would be better to put it in TRAINS for easier modification if you update anything. Also to save the maintenance burden of Keras Tuner. Thanks.

[bmartinn]

Sure @haifeng-jin , one question, could I get notified if the Logger interface changes ?
This will really make things easier on maintenance on our side.
(I'm not sure GitHub has any hooks on interface / commits per specific file, but this would really be useful :)

[haifeng-jin]

@bmartinn I think the only way is to read the release notes.
I don't think the Logger interface would change recently. : )

[bmartinn]

@haifeng-jin initial integration should already be in the text Trains RC :)