Change SurrogateBenchmarkProblem to TorchModelBridge #2591
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D59883218 |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2591 +/- ##
==========================================
- Coverage 95.22% 95.20% -0.02%
==========================================
Files 489 489
Lines 47703 47678 -25
==========================================
- Hits 45426 45394 -32
- Misses 2277 2284 +7 ☔ View full report in Codecov by Sentry. |
|
This pull request was exported from Phabricator. Differential Revision: D59883218 |
dme65
pushed a commit
to dme65/Ax
that referenced
this pull request
Jul 19, 2024
Summary: Pull Request resolved: facebook#2591 This diff changes our surrogate benchmark problems to use a `TorchModelBridge` instead of a `Surrogate` **Why this change?** Our current surrogate benchmark problems are constructed in the transformed space and thus don't have any transform. Constructing the surrogates in the transformed space doesn't play well with things like HSS and also doesn't allow us to fully utilize methods targeting discrete and mixed search spaces. This setup also plays much better with how we normally fit models in Ax as, e.g., cross-validation is done on the model bridge. Going forward, constructing a surrogate benchmark problem will end up looking something like this: 1. Load data 2. Fit an Ax modelbridge 3. Cross-validate the modelbridge to make sure the model fit quality looks good 4. Save the components needed to reconstruct the modelbridge (e.g., state dict) Reviewed By: saitcakmak Differential Revision: D59883218
|
This pull request was exported from Phabricator. Differential Revision: D59883218 |
dme65
pushed a commit
to dme65/Ax
that referenced
this pull request
Jul 19, 2024
Summary: Pull Request resolved: facebook#2591 This diff changes our surrogate benchmark problems to use a `TorchModelBridge` instead of a `Surrogate` **Why this change?** Our current surrogate benchmark problems are constructed in the transformed space and thus don't have any transform. Constructing the surrogates in the transformed space doesn't play well with things like HSS and also doesn't allow us to fully utilize methods targeting discrete and mixed search spaces. This setup also plays much better with how we normally fit models in Ax as, e.g., cross-validation is done on the model bridge. Going forward, constructing a surrogate benchmark problem will end up looking something like this: 1. Load data 2. Fit an Ax modelbridge 3. Cross-validate the modelbridge to make sure the model fit quality looks good 4. Save the components needed to reconstruct the modelbridge (e.g., state dict) Reviewed By: saitcakmak Differential Revision: D59883218
Summary: Pull Request resolved: facebook#2591 This diff changes our surrogate benchmark problems to use a `TorchModelBridge` instead of a `Surrogate` **Why this change?** Our current surrogate benchmark problems are constructed in the transformed space and thus don't have any transform. Constructing the surrogates in the transformed space doesn't play well with things like HSS and also doesn't allow us to fully utilize methods targeting discrete and mixed search spaces. This setup also plays much better with how we normally fit models in Ax as, e.g., cross-validation is done on the model bridge. Going forward, constructing a surrogate benchmark problem will end up looking something like this: 1. Load data 2. Fit an Ax modelbridge 3. Cross-validate the modelbridge to make sure the model fit quality looks good 4. Save the components needed to reconstruct the modelbridge (e.g., state dict) Reviewed By: saitcakmak Differential Revision: D59883218
|
This pull request was exported from Phabricator. Differential Revision: D59883218 |
|
This pull request has been merged in c31d5fd. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This diff changes our surrogate benchmark problems to use a
TorchModelBridgeinstead of aSurrogateWhy this change?
Our current surrogate benchmark problems are constructed in the transformed space and thus don't have any transform. Constructing the surrogates in the transformed space doesn't play well with things like HSS and also doesn't allow us to fully utilize methods targeting discrete and mixed search spaces.
This setup also plays much better with how we normally fit models in Ax as, e.g., cross-validation is done on the model bridge. Going forward, constructing a surrogate benchmark problem will end up looking something like this:
Reviewed By: saitcakmak
Differential Revision: D59883218