Added gamma to LBFGSState

[zaccharieramzi]

This PR adds a new attribute to the LBFGState named tuple, gamma.
This attribute is needed when one wants to use the approximation of the Hessian inverse after the algorithm has run using inv_hessian_product.

Indeed I realized that in this implementation, the initial value of the approximation of the Hessian is recomputed at each iteration, as suggested by Nocedal (something that is not present in scipy to the best of my knowledge and understanding of Fortran).

The application I have in mind for the approximation of the Hessian inverse is https://arxiv.org/abs/2106.00553.

(I didn't write an issue first because it's such a small PR, and I was going to write this part anyway for my own xps)

[zaccharieramzi]

One aspect I am not too sure about though in the arguments of inv_hessian_product is start.
It indeed seems like it's used to perform the matrix multiplication in a certain order, but I don't understand why that's necessary.
Is it because with numerical errors the additions become non-commutative?

[mblondel]

One aspect I am not too sure about though in the arguments of inv_hessian_product is start

We use a circular buffer to maintain the history without lists.

[mblondel]

LGTM, thanks a lot for the contribution

[zaccharieramzi]

I think I don't have access to the logs of the copybara failure, or at least I am not sure how to access them in order to correct the failure.

[mblondel]

Sorry, I overlooked this in my previous review but you shouldn't initialize the array here. Please do

Suggested change

	gamma: Any = jnp.array(1.0)
	gamma: Any = jnp.ndarray

and instead initialize gamma in init_state. CC @froystig

[mblondel]

Sorry I meant

  gamma: jnp.ndarray

[zaccharieramzi]

done