Refactor Surrogates #338
Merged
Refactor Surrogates #338
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR is a first step toward a refactored `Surrogate` layout: * It moves the `exp_rep`-to-`comp_rep` transition point into the surrogates, which now become responsible for handling the transformation and can do it in whatever way they need it (which will also simplify scaling later on). * This cleans up the interface because users / calling classes can now pass data in its canonical form (i.e., `exp_rep`) and do not need to worry about transformations. * This also means we can easily expose the trained surrogates to users for model inspection (e.g., feature importance).
This enables reactivating slots
This PR is a next step toward a lean surrogate class layout: * `Surrogate.(_)posterior` now returns a `Posterior` object * `Surrogate._fit` no longer expects the `SearchSpace` as an argument, which brings us closer to the state that `.fit` and `.posterior` operate on the user/dataframe/context-level while `_posterior` and `_fit` operate on the purely mathematical level. This means that a user who writes their own surrogate class effectively only needs to implement the corresponding mathematical model in the latter two methods. Optional context information that may be required for this implementation (like the dimension index of the `TaskParameter` in the passed `Tensor` object) is now encapsulated into a surrogate-specific `context` object, that can be arbitrarily populated by the surrogate class, but whose logic is now cleanly separated from the actual fitting logic. * Adds a new `GaussianSurrogate` base class for (most our other) models that come with the implicit Gaussian noise assumption and effectively only implement mean and (co-)variance estimation. * Improves and simplifies logic of the `catch_constant_target`, re-enabling slots for `Surrogates`
Preparation for use with sklearn's ColumnTransformer, which spits out arrays
|
@AVHopp, @Scienfitz Finally, the epic is completed 😎 Last gate before merging into main |
AVHopp
approved these changes
Aug 13, 2024
AVHopp
reviewed
Aug 14, 2024
AdrianSosic
commented
Aug 14, 2024
Since the searchspace needs to be stored during fitting anyway (because it is needed by the posterior method), we can simply use a regular attribute access and do not need to pass it via the context method argument
Merged
Scienfitz
reviewed
Aug 28, 2024
Merged
AdrianSosic
force-pushed
the
dev/surrogates
branch
from
01ada2b to
ed32ab8
Compare
Scienfitz
approved these changes
Aug 29, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Completes the surrogate factoring, which extended over #278, #309, #315, #325, #337.
Most important changes
Surrogatemethods likeposteriorandfitcan now operate on dataframes in experimental representation, meaning they can also be exposed directly to the user.Posteriorobject instead of implicitly assuming Gaussian distributions. This paves the way for arbitrary surrogate extensions, such as Bernoulli/Categorical surrogates, etc. At the moment, this introduces an explicit coupling to botorch, which is fine because botorch remains a core dependency and the only backend used for complex surrogate modeling. In the future, this can be further abstracted by introducing our ownPosteriorclass.Surrogatelayout has been refined such that the extractedSurrogateProtocol, which now defines the formal interface for all surrogates, imposes minimal requirements to the user.