Exposing mu to the services (stepsize warmup)

[Tuxonomics]

Summary:

Regarding the parameters of the dual averaging optimization for the stepsize in the warmup, all parameters can be set by the user except mu. For certain models, the initial choice of mu can result in a drastic drop of the stepsize for subsequent iterations. This might lead to extra computation time. As mu is adjusted for each window of the mass matrix adjustment, this can in effect add a significant amount of computation time.

Description:

Currently, mu is set to be log(10 * this->nom_epsilon_) which is hardcoded in the algorithm, and the services stan::services::sample::hmc_static_dense_e_adapt, stan::services::sample::hmc_static_diag_e_adapt, stan::services::sample::hmc_nuts_dense_e_adapt and stan::services::sample::hmc_nuts_diag_e_adapt do not expose the factor of epsilon to the user. This issue would entail

• updating the services API,
• adding the factor as an attribute to stan::mcmc::stepsize_adaptation with the corresponding getter and setter and
• updating the corresponding parts in stan::mcmc::adapt_dense_e_nuts, stan::mcmc::adapt_diag_e_nuts, stan::mcmc::adapt_dense_e_static_hmc and stan::mcmc::adapt_diag_e_static_hmc.

There are also the equivalent algorithms in nuts_classic, static_uniform and xhmc.

The default factor of 10 should be kept.

Additional Information:

Issues in Rstan, Pystan and CmdStan would have to be opened as well.

Current Version:

v2.18.0

[bob-carpenter]

Thanks for submitting this and for explaining it so well. Adding new arguments is a lot easier than removing them. I'm adding the "good first issue" tag not so much because this is simple, but because this issue describes what needs to be done so clearly and it's relatively modular and can be done by following the other parameters being passed.

[avehtari]

For the record, I'm adding links to two relevant discussion threads

• https://discourse.mc-stan.org/t/issue-with-dual-averaging/5995
• https://discourse.mc-stan.org/t/comparing-stans-adaptation-phase-to-that-of-nuts-rs/32259/13