Mixed alternating optimizer

[Balandat]

Summary: Adds an optimizer for settings with mixed (continuous and discrete) variables, that interleaves gradient-based optimization (for fixed discrete variables) with nearest-neighbor search over discrete variables (for fixed continuous variables). This will need more cleanup and testing.

Differential Revision: D48419691

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D48419691

[AlexanderMouton]

options={
            "maxiter": options.get("maxiter_continuous", MAX_ITER_CONT),
            "tol": options.get("tol", CONVERGENCE_TOL),
            "batch_limit": options.get("batch_limit", MAX_BATCH_SIZE),
        },

These options gets passed as part of the options dictionary to scipy's optimize.minimize(...) in botorch.optim.utils.timeout.py:

return optimize.minimize(
            fun=fun,
            x0=x0,
            args=args,
            method=method,
            jac=jac,
            hess=hess,
            hessp=hessp,
            bounds=bounds,
            constraints=constraints,
            tol=tol,
            callback=wrapped_callback,
            options=options,
        )

In other words, tol here is None. To change the tolerance of the lbfgsb minimizer, "gtol" and/or "ftol" should be passed in:

options={
            "maxiter": options.get("maxiter_continuous", MAX_ITER_CONT),
            "gtol": options.get("tol", CONVERGENCE_TOL),
            "ftol": options.get("tol", CONVERGENCE_TOL),
            "batch_limit": options.get("batch_limit", MAX_BATCH_SIZE),
        },

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D48419691