Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Line search method based on majorize-minimize (MM) principle #169

Open
JeffFessler opened this issue Jan 3, 2023 · 0 comments
Open

Line search method based on majorize-minimize (MM) principle #169

JeffFessler opened this issue Jan 3, 2023 · 0 comments

Comments

@JeffFessler
Copy link

I have implemented a line search method based on majorize-minimize (MM) principles for cost functions of the form $f(x) = \sum_j f_j(A_j x)$ where each $f_j$ has a Lipschitz gradient (or tighter quadratic majorizer). It runs much faster than the general purpose line search methods here, but of course it is specific to this family of functions whereas the methods here are general purpose.

Here is an illustration of it for a smoothed version of the LASSO problem that is emblematic of the kind of applications in inverse problems and machine learning that motivated this work, with @benchmark comparisons to most of the line search methods in this package:
https://jefffessler.github.io/MIRT.jl/dev/generated/examples/3-ls-mm/

My question here is whether such a (somewhat) specialized line search algorithm belongs here, or if you'd prefer to keep this package focused on general-purpose search methods. Up to you.

The code itself is here currently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant