Global Sensitivity Analysis for Categorical Features #2357
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D56070326 |
SebastianAment
added a commit
to SebastianAment/Ax
that referenced
this pull request
Apr 12, 2024
Summary: Sobol sensitivity analysis and the sensitivity plots currently do not support categorical features. The plots in particular error out when categorical features are present because the sensitivity analysis (both first order and gradient-based). This commit adds support for categorical features by - introducing a uniformly random integer-valued point distribution for categorical and ordinal features in `SobolSensitivity`, in line with the usual integer-valued encoding, - ignoring the sign of the derivative-based sensitivity analysis of the categorical features, since the `CategoricalKernel` is non-differentiable, and the "direction" of categorical features is not well defined, - adding a separate case for categorical features to the sensitivity analysis plot, stating that the categorical features "affect" but don't "increase" or "decrease" the metric. Note that the results for the categorical features are still first order Sobol indices, so all the results in the plot are on the same scale. Differential Revision: D56070326
SebastianAment
force-pushed
the
export-D56070326
branch
from
e425c26
to
c36dd93
Compare
|
This pull request was exported from Phabricator. Differential Revision: D56070326 |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2357 +/- ##
==========================================
+ Coverage 94.91% 94.92% +0.01%
==========================================
Files 491 495 +4
Lines 47788 48049 +261
==========================================
+ Hits 45356 45609 +253
- Misses 2432 2440 +8 ☔ View full report in Codecov by Sentry. |
SebastianAment
added a commit
to SebastianAment/Ax
that referenced
this pull request
Apr 15, 2024
Summary: Sobol sensitivity analysis and the sensitivity plots currently do not support categorical features. The plots in particular error out when categorical features are present because the sensitivity analysis (both first order and gradient-based). This commit adds support for categorical features by - introducing a uniformly random integer-valued point distribution for categorical and ordinal features in `SobolSensitivity`, in line with the usual integer-valued encoding, - ignoring the sign of the derivative-based sensitivity analysis of the categorical features, since the `CategoricalKernel` is non-differentiable, and the "direction" of categorical features is not well defined, - adding a separate case for categorical features to the sensitivity analysis plot, stating that the categorical features "affect" but don't "increase" or "decrease" the metric. Note that the results for the categorical features are still first order or total Sobol indices, so all the results in the plot are on the same scale. Differential Revision: D56070326
SebastianAment
force-pushed
the
export-D56070326
branch
from
c36dd93
to
2b94086
Compare
|
This pull request was exported from Phabricator. Differential Revision: D56070326 |
Summary: Sobol sensitivity analysis and the sensitivity plots currently do not support categorical features. The plots in particular error out when categorical features are present because the sensitivity analysis (both first order and gradient-based). This commit adds support for categorical features by - introducing a uniformly random integer-valued point distribution for categorical and ordinal features in `SobolSensitivity`, in line with the usual integer-valued encoding, - ignoring the sign of the derivative-based sensitivity analysis of the categorical features, since the `CategoricalKernel` is non-differentiable, and the "direction" of categorical features is not well defined, - adding a separate case for categorical features to the sensitivity analysis plot, stating that the categorical features "affect" but don't "increase" or "decrease" the metric. Note that the results for the categorical features are still first order or total Sobol indices, so all the results in the plot are on the same scale. Differential Revision: D56070326
SebastianAment
force-pushed
the
export-D56070326
branch
from
2b94086
to
bbe1743
Compare
|
This pull request was exported from Phabricator. Differential Revision: D56070326 |
|
This pull request has been merged in 43cef62. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Sobol sensitivity analysis and the sensitivity plots currently do not support categorical features. The plots in particular error out when categorical features are present because the sensitivity analysis (both first order and gradient-based).
This commit adds support for categorical features by
SobolSensitivity, in line with the usual integer-valued encoding,CategoricalKernelis non-differentiable, and the "direction" of categorical features is not well defined,Note that the results for the categorical features are still first order Sobol indices, so all the results in the plot are on the same scale.
Differential Revision: D56070326