Fetching global causal effects and global policy from SDK backend #1753

gaugup · 2022-09-28T23:53:48Z

Description

Currently the causal functionality shows causal effects and policy for the entire data. Selecting a cohort doesn't have an impact of the causal tab. This change adds the functionality to fetch the global causal effects and policy from the SDK backend.

Screenshot of global causal effects requested from SDK backend.

Screenshot of global causal policy requested from SDK backend.

Checklist

I have added screenshots above for all UI changes.
I have added e2e tests for all UI changes.
Documentation was updated if it was needed.

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

codecov-commenter · 2022-09-29T00:14:03Z

Codecov Report

Merging #1753 (0f454aa) into main (86c66a4) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1753   +/-   ##
=======================================
  Coverage   89.32%   89.32%           
=======================================
  Files         118      118           
  Lines        5927     5927           
=======================================
  Hits         5294     5294           
  Misses        633      633

Flag	Coverage Δ
unittests	`89.32% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

github-actions · 2022-09-29T00:30:36Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/GlobalCausalCohortSupport/dashboard/index.html

github-actions · 2022-10-04T17:33:09Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/GlobalCausalCohortSupport/dashboard/index.html

* Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

github-actions · 2022-10-05T20:26:46Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/GlobalCausalCohortSupport/dashboard/index.html

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

github-actions · 2022-11-23T19:12:56Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/GlobalCausalCohortSupport/dashboard/index.html

libs/core-ui/src/lib/Context/ModelAssessmentContext.tsx

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

github-actions · 2022-12-09T02:28:35Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/GlobalCausalCohortSupport/dashboard/index.html

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

github-actions · 2022-12-09T23:24:20Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/GlobalCausalCohortSupport/dashboard/index.html

...salAnalysisDashboard/Controls/CausalAnalysisView/CausalAggregateView/CausalAggregateView.tsx

github-actions · 2022-12-10T02:29:21Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/gaugup/GlobalCausalCohortSupport/dashboard/index.html

) * Add callback for global causal effects Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Query SDK backend to get global causal effects Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix bug in causal serialization Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Adding callback for causal ploicy Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * propagate global and local effects only Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add support for getting global causal effects from SDK backend Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * lint fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix bug Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Modify the cohort message Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Move to state Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove console.log Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change the if condition Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove force update Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Add missing property 'id' in ICounterfactualData Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Update some sample notebooks to include dropped_features (#1836) * update some notebooks to include dropped_features * update some notebooks to include dropped_features * update some notebooks to include dropped_features * remove unused changes * remove unused changes * address comments * upgrade setup-python github action from v2 to v4 and pin to older ubuntu image for python 3.6 to fix build error (#1838) * Gate call to probability distribution end point (#1839) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Revert "Update some sample notebooks to include dropped_features (#1836)" (#1837) This reverts commit d89b03a. * fix cf error on unable to update text (#1842) * Disable partial dependence plots for large data scenarios (#1843) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove redundant call to serialize_json_safe() in raiwidgets causal tests (#1840) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fetching global causal effects and global policy from SDK backend (#1753) * Add callback for global causal effects Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Query SDK backend to get global causal effects Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix bug in causal serialization Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Adding callback for causal ploicy Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * propagate global and local effects only Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add support for getting global causal effects from SDK backend Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * lint fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix bug Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Modify the cohort message Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Move to state Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove console.log Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change the if condition Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove force update Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Update requirements-linting.txt (#1847) * Remove typo casual and correcting it to causal (#1846) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Disable local causal chart for large data support (#1845) * Disable local causal chart for large data support Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * add multilabel text example and fix correct and incorrect groups in data analysis (#1848) * Add number of cohort samples inside generateMetrics() (#1849) * Add number of cohort samples inside generateMetrics() Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Yarn lintfix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * add multilabel RAI vision example for fridge multilabel dataset (#1844) * fix install command for speech to text example (#1856) * Send the correct column name to SDK backend for model overview distribution charts (#1835) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Derive fairness metrics from metric stats instead of cohort stats (#1854) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * add upper bounds to greenlet and gevent in rai-core-flask as 5.0 release fails until they are more stable (#1852) * fix loading older versions of RAIInsights due to _FEATURE_METADATA key error (#1857) * Update maintainers (#1860) * Update requirements.txt in `raiwidgets` and `responsibleai` to pin numpy upper bound (#1864) * Update requirements.txt * Update requirements.txt * Port `cohort.py` from `raiwidgets` to `raiutils` (#1859) * Port cohort.py from raiwidgets to raiutils Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more constants Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add constants.py Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make constants public Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add common APIs for model overview metrics (#1862) * Add metric SDK APIs for Model Overview metrics Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * remove console.log() Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Port `cohort.py` from `raiwidgets` to `raiutils` (#1859) * Port cohort.py from raiwidgets to raiutils Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more constants Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add constants.py Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make constants public Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * lint fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Update requirements-linting.txt (#1861) * update raiwidgets to rai-core-flask v0.5.0 (#1851) * fix error in erroranalysis heatmap caused by max value over right side of max bin (#1866) * remove dead space in aggregate feature importance (#1869) * display tree branching decisions (#1870) * Remove warning for 'dropped_features are not in use currently.' (#1871) * display tree branching decisions * Remove warning for 'dropped_features are not in use currently.' * turn off fail-fast for python tests (#1878) * add multilabel model overview metrics (#1882) * Show branching decisions on hover over on unselected tree nodes (#1881) * show tree label on hover over * address comments * Update some sample notebooks to include dropped_features (#1836) * update some notebooks to include dropped_features * update some notebooks to include dropped_features * update some notebooks to include dropped_features * remove unused changes * remove unused changes * address comments * Revert "Update some sample notebooks to include dropped_features (#1836)" (#1837) This reverts commit d89b03a. * Update requirements-linting.txt (#1847) * Add number of cohort samples inside generateMetrics() (#1849) * Add number of cohort samples inside generateMetrics() Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Yarn lintfix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Update requirements-linting.txt (#1861) * add multilabel model overview metrics (#1882) * Update `raiutils` in `raiwidgets`, `responsibleai`, `erroranalysis` and `nlp_feature_extractors` (#1873) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Undo changes Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: tongy-msft <91754176+tongyu-microsoft@users.noreply.github.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com>

gaugup added 22 commits September 16, 2022 09:50

Add callback for global causal effects

8a0849c

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Query SDK backend to get global causal effects

8855ee4

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Fix bug in causal serialization

4bcd19a

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Adding callback for causal ploicy

a16076b

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

0d18277

propagate global and local effects only

91f0978

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Add support for getting global causal effects from SDK backend

c48428b

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

lint fix

480a504

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Fix lint

a6f5b73

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Fix

1cbd6ba

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

1b79814

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

9069c87

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

2980526

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

e51cc24

Fix bug

4602229

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Modify the cohort message

09449b8

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

aae444b

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

697638f

Move to state

c06ff2a

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

1c599b4

Fix imports

de36f42

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Remove console.log

8e40107

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

gaugup requested review from imatiach-msft, tongyu-microsoft, xuke444, RubyZ10, vinuthakaranth and romanlutz as code owners September 28, 2022 23:53

gaugup added 2 commits October 3, 2022 15:32

Merge branch 'main' into postga

a54a531

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

1015233

gaugup and others added 2 commits October 5, 2022 12:44

Merge branch 'postga' into gaugup/GlobalCausalCohortSupport

7c337a0

gaugup changed the base branch from postga to main November 21, 2022 18:22

gaugup requested a review from hawestra as a code owner November 21, 2022 18:22

gaugup added 3 commits November 21, 2022 10:24

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

f2ca1c1

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

63c8d67

Fix lint

dfb8fb2

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

romanlutz reviewed Dec 6, 2022

View reviewed changes

libs/core-ui/src/lib/Context/ModelAssessmentContext.tsx Show resolved Hide resolved

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

2238b19

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

gaugup added 4 commits December 9, 2022 12:18

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

eea017f

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

e73d57d

Remove force update

1aea750

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Fix lint

dafa93c

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

xuke444 reviewed Dec 9, 2022

View reviewed changes

...salAnalysisDashboard/Controls/CausalAnalysisView/CausalAggregateView/CausalAggregateView.tsx Show resolved Hide resolved

xuke444 approved these changes Dec 9, 2022

View reviewed changes

Merge branch 'main' into gaugup/GlobalCausalCohortSupport

0f454aa

gaugup enabled auto-merge (squash) December 10, 2022 01:58

gaugup merged commit 0b89381 into main Dec 10, 2022

gaugup deleted the gaugup/GlobalCausalCohortSupport branch December 10, 2022 03:28

imatiach-msft mentioned this pull request Jan 6, 2023

release raiwidgets and responsibleai 0.24.0 #1891

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetching global causal effects and global policy from SDK backend #1753

Fetching global causal effects and global policy from SDK backend #1753

gaugup commented Sep 28, 2022 •

edited

Loading

codecov-commenter commented Sep 29, 2022 •

edited

Loading

github-actions bot commented Sep 29, 2022

github-actions bot commented Oct 4, 2022

github-actions bot commented Oct 5, 2022

github-actions bot commented Nov 23, 2022

github-actions bot commented Dec 9, 2022

github-actions bot commented Dec 9, 2022

github-actions bot commented Dec 10, 2022

Fetching global causal effects and global policy from SDK backend #1753

Fetching global causal effects and global policy from SDK backend #1753

Conversation

gaugup commented Sep 28, 2022 • edited Loading

Description

Checklist

codecov-commenter commented Sep 29, 2022 • edited Loading

Codecov Report

github-actions bot commented Sep 29, 2022

github-actions bot commented Oct 4, 2022

github-actions bot commented Oct 5, 2022

github-actions bot commented Nov 23, 2022

github-actions bot commented Dec 9, 2022

github-actions bot commented Dec 9, 2022

github-actions bot commented Dec 10, 2022

gaugup commented Sep 28, 2022 •

edited

Loading

codecov-commenter commented Sep 29, 2022 •

edited

Loading