Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/3149 jacobian optimize #3152

Merged
merged 8 commits into from
Nov 15, 2022
Merged

Conversation

bob-carpenter
Copy link
Contributor

Submission Checklist

  • Run unit tests: ./runTests.py src/test/unit
  • Run cpplint: make cpplint [can't figure out how to do this as it hardcodes python and I need to use python3 on Mac OS X and it doesn't seem to take aliases or a specification of PYTHON2 environment variable]
  • Declare copyright holder and open-source license: see below

Summary

Add a template parameter to all optimization functions (Newton, BFGS, L-BFGS) to indicate whether or not to include Jacobian adjustments. Until now, we had hard coded turning the Jacobian off to provide maximum likelihood estimates. With the Jacobian turned on, we will now support an option for max a posteriori (MAP) estimates.

Intended Effect

Use MAP estimates in Laplace approximation.

How to Verify

Unit tests with a new model with the Jacobian calculated and optimized explicitly.

Side Effects

Shouldn't be any because I made it the last argument and gave it a default of false so that it falls back on existing behavior.

Documentation

Yes, in code.

Copyright and Licensing

Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company):

Simons Foundation

By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

Copy link
Member

@mitzimorris mitzimorris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@avehtari
Copy link
Collaborator

I missed this one earlier, too

we will now support an option for max a posteriori (MAP) estimates.

It would be good to be explicit whether the MAP is in unconstrained or constrained space (as these are not in general the same)

@WardBrian WardBrian deleted the feature/3149-jacobian-optimize branch November 18, 2022 14:07
@bob-carpenter
Copy link
Contributor Author

We compute the optimization in the unconstrained space using the Jacobian adjustment. We then map the posterior mode in the unconstrained space back to the constrained space.

Then for the Laplace approximation, we map back from constrained to unconstrained, lay down the normal approximation in the unconstrained space (including Jacobian for the MAP option), sample from the approximate normal in the unconstrained space, and then inverse transform the draws back to the constrained space.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants