Send the correct column name to SDK backend for model overview distribution charts #1835
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…bution charts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
gaugup
requested review from
imatiach-msft,
tongyu-microsoft,
xuke444,
RubyZ10,
vinuthakaranth,
romanlutz and
hawestra
as code owners
Codecov Report
@@ Coverage Diff @@
## main #1835 +/- ##
==========================================
+ Coverage 79.00% 88.19% +9.19%
==========================================
Files 19 52 +33
Lines 1324 1923 +599
==========================================
+ Hits 1046 1696 +650
+ Misses 278 227 -51
Flags with carried forward coverage won't be shown. Click here to find out more. Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
|
I am somewhat confused by the description. How did it work so far? You write that it "enables the box plot for regression" which worked just fine as of a few days ago. I seem to be missing something. Just taking a quick glance at the code I thought that sending the id of the option is entirely intended as it's generally preferable to use IDs than user-provided text for identifying anything. |
xuke444
reviewed
Dec 9, 2022
.../src/lib/ModelAssessmentDashboard/Controls/ModelOverview/ProbabilityDistributionBoxChart.tsx
Show resolved
Hide resolved
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
By working for regression, I meant that it works for regression by getting the distribution from SDK backend. For ID, please see my response on Ke's comment. |
xuke444
approved these changes
Dec 15, 2022
1 similar comment
gaugup
added a commit
that referenced
this pull request
Dec 19, 2022
…bution charts (#1835) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
gaugup
added a commit
that referenced
this pull request
Jan 6, 2023
…bution charts (#1835) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
3 tasks
gaugup
added a commit
that referenced
this pull request
Jan 6, 2023
* Add missing property 'id' in ICounterfactualData Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Update some sample notebooks to include dropped_features (#1836) * update some notebooks to include dropped_features * update some notebooks to include dropped_features * update some notebooks to include dropped_features * remove unused changes * remove unused changes * address comments * upgrade setup-python github action from v2 to v4 and pin to older ubuntu image for python 3.6 to fix build error (#1838) * Gate call to probability distribution end point (#1839) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Revert "Update some sample notebooks to include dropped_features (#1836)" (#1837) This reverts commit d89b03a. * fix cf error on unable to update text (#1842) * Disable partial dependence plots for large data scenarios (#1843) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove redundant call to serialize_json_safe() in raiwidgets causal tests (#1840) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fetching global causal effects and global policy from SDK backend (#1753) * Add callback for global causal effects Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Query SDK backend to get global causal effects Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix bug in causal serialization Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Adding callback for causal ploicy Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * propagate global and local effects only Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add support for getting global causal effects from SDK backend Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * lint fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix bug Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Modify the cohort message Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Move to state Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove console.log Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change the if condition Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove force update Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Update requirements-linting.txt (#1847) * Remove typo casual and correcting it to causal (#1846) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Disable local causal chart for large data support (#1845) * Disable local causal chart for large data support Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * add multilabel text example and fix correct and incorrect groups in data analysis (#1848) * Add number of cohort samples inside generateMetrics() (#1849) * Add number of cohort samples inside generateMetrics() Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Yarn lintfix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * add multilabel RAI vision example for fridge multilabel dataset (#1844) * fix install command for speech to text example (#1856) * Send the correct column name to SDK backend for model overview distribution charts (#1835) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Derive fairness metrics from metric stats instead of cohort stats (#1854) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * add upper bounds to greenlet and gevent in rai-core-flask as 5.0 release fails until they are more stable (#1852) * fix loading older versions of RAIInsights due to _FEATURE_METADATA key error (#1857) * Update maintainers (#1860) * Update requirements.txt in `raiwidgets` and `responsibleai` to pin numpy upper bound (#1864) * Update requirements.txt * Update requirements.txt * Port `cohort.py` from `raiwidgets` to `raiutils` (#1859) * Port cohort.py from raiwidgets to raiutils Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more constants Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add constants.py Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make constants public Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add common APIs for model overview metrics (#1862) * Add metric SDK APIs for Model Overview metrics Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * remove console.log() Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Port `cohort.py` from `raiwidgets` to `raiutils` (#1859) * Port cohort.py from raiwidgets to raiutils Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more constants Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add constants.py Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make constants public Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * lint fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Update requirements-linting.txt (#1861) * update raiwidgets to rai-core-flask v0.5.0 (#1851) * fix error in erroranalysis heatmap caused by max value over right side of max bin (#1866) * remove dead space in aggregate feature importance (#1869) * display tree branching decisions (#1870) * Remove warning for 'dropped_features are not in use currently.' (#1871) * display tree branching decisions * Remove warning for 'dropped_features are not in use currently.' * turn off fail-fast for python tests (#1878) * add multilabel model overview metrics (#1882) * Show branching decisions on hover over on unselected tree nodes (#1881) * show tree label on hover over * address comments * Update some sample notebooks to include dropped_features (#1836) * update some notebooks to include dropped_features * update some notebooks to include dropped_features * update some notebooks to include dropped_features * remove unused changes * remove unused changes * address comments * Revert "Update some sample notebooks to include dropped_features (#1836)" (#1837) This reverts commit d89b03a. * Update requirements-linting.txt (#1847) * Add number of cohort samples inside generateMetrics() (#1849) * Add number of cohort samples inside generateMetrics() Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Yarn lintfix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Update requirements-linting.txt (#1861) * add multilabel model overview metrics (#1882) * Update `raiutils` in `raiwidgets`, `responsibleai`, `erroranalysis` and `nlp_feature_extractors` (#1873) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Undo changes Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: tongy-msft <91754176+tongyu-microsoft@users.noreply.github.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
gaugup
added a commit
that referenced
this pull request
Jan 9, 2023
* Refactor metric computation in ModelOverview Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Move metrics to state management Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * add multilabel RAI vision example for fridge multilabel dataset (#1844) * fix install command for speech to text example (#1856) * Send the correct column name to SDK backend for model overview distribution charts (#1835) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Derive fairness metrics from metric stats instead of cohort stats (#1854) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * add upper bounds to greenlet and gevent in rai-core-flask as 5.0 release fails until they are more stable (#1852) * fix loading older versions of RAIInsights due to _FEATURE_METADATA key error (#1857) * Fix build Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort updation and deletion logic Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add delete logic for selected cohort index Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments part-2 Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove unnecessary condition Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
It seems we were sending a number as a class which was causing an exception. Now sending the correct axis via "text" field. This PR also enables the box plot for regression.
classification bug fix.
regression support.
Checklist