Save training loss history to .csv file

[kxk302]

Enable saving training loss history to .csv file (From https://help.galaxyproject.org/t/csvlogger-callback-with-create-deep-learning-model/10447)

Hi,

I am using the Create deep learning model tool (with an optimizer, loss function and fit parameters) and I would like to get an indication about the convergence of the loss function as a function of the epoch.

I thought that information would come in the CSVLogger but that does not seem to be the case, is this the correct option?

Can someone point me to where I can find the training loss history (and not simply the final value)?

J34ni

[qiagu]

@kxk302 Thanks for reposting the issue here. It would be great to see some example code and reproduce the problem.

[kxk302]

Hi @qiagu,

I think what the user is asking for is having the loss function value as a function of epoch number. For example, if the number of epochs is 10, and the loss function for the model is cross entropy, they expect to see 10 cross entropy values for the 10 epochs, which are hopefully improving as the training goes on.

CSVLogger (https://keras.io/api/callbacks/csv_logger/) provides for a callback mechanism that streams epoch results to a CSV file. Here is an example code snippet to setup CSVLogger (from the link I shared):

csv_logger = CSVLogger('training.log')
model.fit(X_train, Y_train, callbacks=[csv_logger])

However, when I look at the code for keras_train_and_eval.py (https://github.com/bgruening/galaxytools/blob/master/tools/sklearn/keras_train_and_eval.py), I do not see CSVLogger being setup. I see the fit() method being called on these lines and in none of them CSVLogger is being set.

https://github.com/bgruening/galaxytools/blob/master/tools/sklearn/keras_train_and_eval.py#L452
https://github.com/bgruening/galaxytools/blob/master/tools/sklearn/keras_train_and_eval.py#L454
https://github.com/bgruening/galaxytools/blob/master/tools/sklearn/keras_train_and_eval.py#L456

I think we just need to setup CSVLogger in keras_train_and_eval.py to provide the functionality the user is asking for. I'll modify keras_train_and_eval.py to see if this works. Let me know your thoughts/suggestions.

[qiagu]

Hi @kxk302,

Thanks for looking into the user's question and also the Galaxy-ml source code. The CSVLogger was supported in the early versions of Galaxy-ml. However, it got dropped off later and I can't recall why that happened. :(