Iteration count and AbstractLogger interface

[PhilipVinc]

When we implement the AbstractLogger interface, we need to find a way to pass the step (iteration) count to tensorboard. This can be done nicely by passing it as a key-value pair in the logged message, but I am interested in the case where this is not done.

The obvious solution is to store a global iteration count in the logger itself, but how do we update it? Defining a set_step would make the code break if one changes Logger. Maybe if one logs step this will not be logged and just used to update the internal counter?

[oxinabox]

Magic named arguments are a thing loggers do.
Eg. ConsoleLogger handled max_log to control the maximum number of times a given log message will be printed.

I think we shoud have an internal step to the logger.
I think it should increment every time you log,
unless you pass a samestep=true argument. (or similar).

I think always increment is a good default,
since if you have a collection of logs that are supposed to be at the same time
then you should write:

@info "Foo" x=2 y=3 z=1

or if you need to break them up

@info "Foo" x=2
@info "Foo" y=3 samestep=true
@info "Foo" z=1 samestep=true

But if you break them up, and forget to set samestep,
then that is fine, you can read tensorboard in WallTime mode and it will still all look right.

I don't think directly being able to set the step is useful since why would you set it to any particular number?
Possible exception is to reset it to zero, but I feel like this is better handled by constructing a new logger.
Or perhaps by manipulating the logger object.

[shashikdm]

I think there should be option to set step. Increment step is a good default. But user may need to give specific step numbers to get precise line graphs such as logging loss after every 10 epochs.

[PhilipVinc]

I agree that there should be some way to skip multiple steps, as it does fit to a common use-case I have (comparing convergence of different algorithms, with different runtimes).

Though, I would keep as default behaviour that of incrementing the step, as Lyndon suggested, as it works well with solvers implementing the Iterative interface (something I'm keen of).

A delta_step magic argument might combine both solutions. delta_step=0 would not increase the step, and if one wants to skip a certain number of steps he could just change it.

Potentially we could also have a step magic argument, though I'm not sure we would really need.

[oxinabox]

Perhaps step_increment defaulting to 1. (delta does not imply direction).
And if you want to abolutely set the step then you can deal with the logger object itself.

Maybe log_step_incrment to make name collision even more unlikely.

[PhilipVinc]

It's a bit verbose but log_step_increment makes sense.
Yeah, defaulting to 1 so we get the behaviour you described is the best idea.

I'll implement this into #9 branch which also solves #5 .