Feature Requests: Inverse Problem Solvers

[kecnry]

Please include any suggestions for extensions to the inverse problem solvers in PHOEBE here, including:

• new algorithms or codes to include
• new capabilities or options to existing algorithms
• new tutorials or example scripts

Bug reports to existing capabilities belong as a new issue instead. For questions or discussion about existing features, please start a new discussion.

For an overview of the existing capabilities, see:

• PHOEBE 2.3 release paper
• Tutorial: Inverse Problem Overview (and the advanced tutorials linked from the bottom of that page)
• Tutorial: Distributions
• Tutorial: Gaussian Processes

Each individual suggestion/feature should be its own top-level comment so that they can easily be tracked and "voted" to see which have the most interest. Any conversation about that feature or progress with implementation and/or testing should be as replies to those threads.

[kecnry]

Allow for a callable custom_lnpriors_callable in addition to the existing custom_lnprobability_callable when calling b.run_solver. Although this can currently be handled within custom_lnprobability_callable, the current implementation does not allow rejecting a sample before running the forward model, which could be useful in some cases.

[kecnry]

Convergence stopping criteria for emcee solver. Currently the chains and per-chain autocorrelation times are exposed to the user and it is completely up to them to decide how to determine if the chains have converged sufficiently (and what values of thin, burnin, and lnprob_cutoff are appropriate).

This emcee tutorial provides a nice overview of some of the options we may want to consider.

[kecnry]

plotting the autocorrelation function has been implemented in 1d92e69 (currently in the development branch) and should be included in the 2.4 release.

• implements phoebe.helpers.process_acf, process_acf_from_solution, acf, acf_ci, acf_ci_bartlett
• implements style='acf' and 'acf_lnprobabilities' when plotting emcee solutions

[kecnry]

Samplers: ability to expose "warnings" during sampling. Currently these are only available in the log if the appropriate logger has been initialized. Options would be to:

• expose in a corner plot similar to style='failed' (either in separate corner plot or lumped in with errors but with a different marker?)
• ability to optionally treat warnings as errors and reject them during sampling (would this be across the board or have the ability to choose which warnings would be rejected). As is, this technically could be done manually through either custom_lnprobability_callable (or custom_lnpriors_callable (see Feature Requests: Inverse Problem Solvers #407 (comment)) by calling b.run_checks within the custom function... but that would be quite advanced and not exactly convenient.

[kecnry]

New Sampler: importance sampling (see snowline, for example)

[kecnry]

New Samplers: other mcmc implementations besides emcee (pymc3, autoemcee)

[kecnry]

MCMC automated "branch finder" - to automatically find and isolate the best branch from an MCMC run in use with resampling

[kecnry]

Automated backend transitioning - so a solver could occasionally test the residuals between two different backends and transition to more expensive models as needed. This would need some serious testing and probably would not allow adding new parameters (degrees of freedom) within a single run.

[kecnry]

Detrending or data-preprocessing plugins. In theory, GPs "should" be used to create a noise-model, but this isn't always practical and noisy data can cause issues with estimators (where you don't have a decent physical model yet).

[kecnry]

Bootstrapping algorithms as an alternative method (to mcmc or nested sampling "posteriors") to determine uncertainties

[jsinkbaek]

I realize that this is an old comment, but wanted to add a take.

With the current state of computation speed with PHOEBE, these will be expensive since the optimizers usually require a lot of evaluations still.

They have potential use for testing the assumption that the data only includes white noise. If a residual block bootstrap is implemented, the residuals can be shuffled in time/phase in groups to test whether they are biasing the fit.

[kecnry]

Main sequence "constraint" (possibly via a "prior" on the relationship between fundamental stellar parameters rather than an exact constraint between parameters).

[kecnry]

Additional "observables" or "geometric properties" which could then be included in the likelihood (either by placing priors or by allowing the observed values with uncertainties to be in the cost-function):

• eclipse width/depth/separation (also useful to put a prior on requiring or forbidding presence of eclipse)
• light-ratio (time-dependent or marginalized over orbit? Or just addressed with per-component fluxes exposed in the light curve?)
• SED or colors (time-dependent or marginalized over orbit?)
• flux-weighted loggs (time-dependent or marginalized over orbit?)
• observed temperatures (time-dependent or marginalized over orbit?) - including contributions from spots, distortion, irradiation, etc.

Many of these could, in theory, currently be handled manually with custom_lnprobability_callable.

[kecnry]

See #571 and #662 for use-cases where this is also needed to access coupled pblum values.

[kecnry]

See also #919 for a case of accessing "observed" temperatures.

[kecnry]

GPs: test or prevent GPs from "stealing" signal from the binary model.

[aprsa]

Differential corrections. While many scorn at DC, it is demonstrably the fastest way to get to a minimum provided the starting point is reasonably close. DC should be straight-forward to implement and, when used correctly, it would add value to the fitting framework.

[kecnry]

DC was implemented and included in the 2.4 release.

[kecnry]

When plotting MCMC trace plots, allow coloring by lnprobabilities. When plotting lnprobabilities vs iteration, allow coloring by any single parameter value.

[kecnry]

This has been implemented in f313775 (currently in the development branch) and will be included in the 2.4 release.

• if style='lnprobability': c can be a twig pointing to any of the sampled parameters (whether within adopt_parameters or not)
• if style='trace': c can be a twig or 'lnprobability'
• NOTE: adding color to lineplots with many walkers/iterations can become expensive

[kecnry]

emcee: allow setting init_from to an existing emcee solution instead of requiring calling adopt_solution as an intermediate step. See the resampling tutorial for the current requirement to call adopt_solution to create a tagged distribution first.

[kecnry]

fake progress_every_niters support in optimizers by (internally) setting a lower maxiter, saving the results, and then restarting at the final position until reaching the full provided maxiter or canceled. This would then allow monitoring the progress and would be useful in cases where the desired xatol for fatol (in the case of nelder-mead) is hard to estimate.. but would likely come with some overhead.

[kecnry]

This has now been implemented in aa40e84 (currently in the development branch). Restarting wasn't needed - instead the current position of the optimizer is accessed and exported. The only overhead (besides I/O) are additional calls to compute the current lnprobability if expose_lnprobabilities == True at each export.

[kecnry]

New Samplers: UltraNest nested sampling (see #414)

[kecnry]

Optimizers: expose "uncertainty estimates" to aid in creating initializing distribution widths for samplers

[kecnry]

New optimizers: lmfit, scipy.optimize.least_squares

[kecnry]

New estimator: phase-dispersion minimization

[jsinkbaek]

It would be interesting to include a bayesian optimization or variational inference algorithm, as a trade-off between wanting a global optimizer (with minimal function evaluations), and approximating the probability distribution.