Implement SoftAbs metric #2242
Description
The SoftAbs metric was first introduced in Betancourt's paper A General Metric for Riemannian Manifold Hamiltonian Monte Carlo, to be used instead of Euclidean metrics. Normal metrics cannot be used because:
This metric quickly runs into problems, however, when the target distribution is not globally convex. In neighborhoods where the Hessian is not positive-definite, for example, the conditional density π(p|q) becomes improper.
Since the Fischer-Rao metric is also not practical, a new metric is introduced:
[exp(αX) + exp(-αX)] · X · [exp(αX) − exp(− αX)]^{-1}
It is further mentioned:
Applying the SoftAbs map to the Hessian guarantees a well-behaved metric for RMHMC, ≀H≀, that preserves the desired properties of the Hessian while regularizing its numerical singularities. In a practical implementation, α limits the scaling of the integration step-size and restrains the integrator from unwise extrapolations, emulating a trust region common in nonlinear optimisation
By implementing the SoftAbs metric we are building a crucial part of RMHMC (#2240), and it may also likely come in handy for other versions of HMC. A good place to start is by looking at this repo.