update manuals for Laplace approximation, optimization allows Jacobian adjustment #606
Conversation
|
Do we need a citation for Laplace approx for Ref manual, and if so, what's a good one? |
| The `laplace` method produces a sample from a Laplace approximation | ||
| centered at the mode of a distribution. | ||
| If the mode is a maximum a posteriori (MAP) estimate, | ||
| the samples provides an estimate of the standard deviation |
There was a problem hiding this comment.
It would be good to explicitly mention here the effect of the constraints. If there are constrained parameters, the Laplace approximation works best in the unconstrained space. The normal approximation is then centered at the mode in the unconstrained space, but the implemented method transforms the normal approximation sample to the constrained space before writing them to csv. Due to the transformation, the samples can provide more than just the standard deviation estimate.
There was a problem hiding this comment.
this discussion should go in the reference manual algorithms section, and we should link to that from here.
I'm confused by what you say - the laplace_sample algorithm always computes on the unconstrained space, and outputs on the constrained scale - that is the way that all the Stan inference algorithms work, so why mention this?
Due to the transformation, the samples can provide more than just the standard deviation estimate.
what additional inforamation is provided? a better estimate of the mode as well?
There was a problem hiding this comment.
apologies - my confusion here is that while the stan::services method laplace_sample takes its inputs on the unconstrained scale, the CmdStan interface takes its inputs on the constrained scale, and then does the unconstrain before calling the laplace_sample method. this is an implementation detail that the user doesn't need to know about.
There was a problem hiding this comment.
added discussion of this to the Laplace section in the reference manual.
There was a problem hiding this comment.
I would still change this also here, e.g.
"The laplace method produces a sample from a normal approximation
centered at the mode of a distribution in the unconstrained space.
If the mode is a maximum a posteriori (MAP) estimate,
the samples provide an estimate of the mean and standard deviation
of the posterior distribution."
In case of constraints, in general the posterior mode in the unconstrained space doesn't correspond to the mean (nor mode) in the constraint space, and thus the sample is needed also to infer the mean.
|
thanks for the careful review - changes made - ready for re-review |
| The `laplace` method produces a sample from a Laplace approximation | ||
| centered at the mode of a distribution. | ||
| If the mode is a maximum a posteriori (MAP) estimate, | ||
| the samples provides an estimate of the standard deviation |
There was a problem hiding this comment.
I would still change this also here, e.g.
"The laplace method produces a sample from a normal approximation
centered at the mode of a distribution in the unconstrained space.
If the mode is a maximum a posteriori (MAP) estimate,
the samples provide an estimate of the mean and standard deviation
of the posterior distribution."
In case of constraints, in general the posterior mode in the unconstrained space doesn't correspond to the mean (nor mode) in the constraint space, and thus the sample is needed also to infer the mean.
|
made all suggested changes. |
Submission Checklist
`r since("VERSION")`Summary
Updated the CmdStan and Reference Manual to reflect changes made in PRs stan-dev/cmdstan#1134, stan-dev/stan#3148, and stan-dev/stan#3152
Issues: #596 and #595
Copyright and Licensing
Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company): Columbia University
By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses: