Finite differences: shouldn't we count f_calls?

[pkofod]

Shouldn't we count the function evaluations done in finite differences towards f_calls?

[johnmyleswhite]

Yes, but we can't ask that library to do that. We should still be able to do it by assuming it's 2 * n where n = length(x) and x is the vector of parameters being optimized over.

[KristofferC]

It is sort of possible to have the library do it by passing in a closure that modifies the variable when it is called but the performance for this has been quite shaky lately on master.

[johnmyleswhite]

That's technically possible but seems to needlessly complicated the finite differencing code.

[pkofod]

I was just thinking about the 2*n solution. Is there any way it can be something else?

[johnmyleswhite]

Only if we don't specify that we want central differencing and the default were to change.

[pkofod]

Okay, I guess we'll just have to be careful then :) I guess we do need some sort of flag though, to pass through to the optimizer, so it knows if it is using finite differences or not (we could show a gradient: user provided, finite differences, or automatic differentiation line in the output as well).

[cortner]

Aren't there techniques to compute multiple finite difference at the same time, if the sparsity structure is known? Maybe better to fix the packages that perform the fds?