Confusion Matrix #1794

t-wangjohn · 2022-10-27T21:19:07Z

Confusion Matrix added in model assessment for multiclass and binary classification

Description

binary classification (light mode)

binary classification (dark mode)

multiclass classification (light mode)

multiclass classification (dark mode)

Too few classes selected error (there's another one with at most 20, but I don't have a dataset handy that triggers the error):

Checklist

I have added screenshots above for all UI changes.
I have added e2e tests for all UI changes.
Documentation was updated if it was needed.

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

…ortance (#1770) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Port tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix rai_insights Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Simply tests in test_cohort_filter.py Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change test name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

codecov-commenter · 2022-10-27T21:49:00Z

Codecov Report

Merging #1794 (56c7ab8) into main (902a4ac) will decrease coverage by 2.71%.
The diff coverage is n/a.

❗ Current head 56c7ab8 differs from pull request most recent head 895f595. Consider uploading reports for the commit 895f595 to get more accurate results

@@            Coverage Diff             @@
##             main    #1794      +/-   ##
==========================================
- Coverage   93.35%   90.64%   -2.72%     
==========================================
  Files          93       46      -47     
  Lines        4574     1881    -2693     
==========================================
- Hits         4270     1705    -2565     
+ Misses        304      176     -128

Flag	Coverage Δ
unittests	`90.64% <ø> (-2.72%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
responsibleai/responsibleai/_config/__init__.py
responsibleai/responsibleai/_config/base_config.py
...ibleai/_tools/shared/state_directory_management.py
...leai/responsibleai/databalanceanalysis/__init__.py
responsibleai/responsibleai/_internal/__init__.py
...sibleai/responsibleai/rai_insights/rai_insights.py
responsibleai/responsibleai/exceptions.py
...tabalanceanalysis/distribution_balance_measures.py
responsibleai/responsibleai/managers/__init__.py
responsibleai/responsibleai/_internal/constants.py
... and 37 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

github-actions · 2022-10-27T21:56:00Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

github-actions · 2022-10-27T22:36:56Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

github-actions · 2022-10-28T16:43:18Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

This is done to solve the cyclomatic complexity es-lint issue.

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

github-actions · 2022-11-21T19:56:32Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

tongyu-microsoft · 2023-01-10T17:35:19Z

Could you confirm that it looks good for large screens?

romanlutz · 2023-01-10T21:43:36Z

Could you confirm that it looks good for large screens?

Example with very large and very small screen: (matches design)

…olbox into t-wangjohn/ConfusionMatrix

github-actions · 2023-01-11T02:02:21Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

…soft/responsible-ai-toolbox into t-wangjohn/ConfusionMatrix

github-actions · 2023-01-11T17:51:57Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

github-actions · 2023-01-11T17:53:24Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

github-actions · 2023-01-11T18:27:17Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

github-actions · 2023-01-11T21:47:11Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

github-actions · 2023-01-11T21:52:41Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

...sessment/src/lib/ModelAssessmentDashboard/Controls/ModelOverview/ModelOverviewChartPivot.tsx

github-actions · 2023-01-12T05:49:45Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/t-wangjohn/ConfusionMatrix/dashboard/index.html

gaugup and others added 9 commits September 29, 2022 16:27

Add postga build trigger (#1755) (#1756)

6f16591

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Merge branch 'main' into postga

a54a531

Merge branch 'main' into postga

8395b89

Merge branch 'main' into postga

c400c30

Change description of cohort selection panel in Aggregate Feature Imp…

d0d338e

…ortance (#1770) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Support cohort filtering of string target in rai_insights (#1771)

18af5ab

* Port tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix rai_insights Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Simplify tests in test_cohort_filter.py (#1772)

c23be59

* Simply tests in test_cohort_filter.py Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change test name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Add Confusion Matrix Heatmap

0983f6b

t-wangjohn requested review from imatiach-msft, tongyu-microsoft, xuke444, RubyZ10, vinuthakaranth and romanlutz as code owners October 27, 2022 21:19

t-wangjohn force-pushed the t-wangjohn/ConfusionMatrix branch from 964e4a6 to 19f6cc3 Compare October 27, 2022 21:53

t-wangjohn requested a review from gaugup October 28, 2022 14:35

t-wangjohn self-assigned this Oct 28, 2022

t-wangjohn force-pushed the t-wangjohn/ConfusionMatrix branch from 19f6cc3 to 4022cb4 Compare October 28, 2022 16:13

Reconfigure Pivots for Model Overview

a362704

This is done to solve the cyclomatic complexity es-lint issue.

t-wangjohn force-pushed the t-wangjohn/ConfusionMatrix branch from 4022cb4 to a362704 Compare November 18, 2022 15:53

t-wangjohn requested a review from hawestra as a code owner November 18, 2022 15:53

t-wangjohn changed the base branch from postga to main November 18, 2022 20:44

t-wangjohn requested a review from kicha0 as a code owner November 18, 2022 20:44

Merge branch 'main' into t-wangjohn/ConfusionMatrix

ea1a9d1

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

romanlutz added 3 commits January 10, 2023 15:59

fix e2e tests

9d77b33

lintfix

5e6af66

lintfix

2a30004

romanlutz added 3 commits January 10, 2023 16:44

Merge branch 'main' into t-wangjohn/ConfusionMatrix

1d71d42

Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…

34dbe6d

…olbox into t-wangjohn/ConfusionMatrix

fix build error

9b4085d

romanlutz added 4 commits January 11, 2023 12:14

fix model overview metric chart assertions

7d6621f

Merge branch 'main' into t-wangjohn/ConfusionMatrix

463e7fa

remove wait

1ac0cb4

Merge branch 't-wangjohn/ConfusionMatrix' of https://github.com/micro…

aa20456

…soft/responsible-ai-toolbox into t-wangjohn/ConfusionMatrix

romanlutz added 3 commits January 11, 2023 15:55

allow wait for this one file

e10b8e6

lintfix

110d568

Merge branch 'main' into t-wangjohn/ConfusionMatrix

a23a94c

JingyaChen approved these changes Jan 11, 2023

View reviewed changes

gaugup reviewed Jan 11, 2023

View reviewed changes

...sessment/src/lib/ModelAssessmentDashboard/Controls/ModelOverview/ModelOverviewChartPivot.tsx Outdated Show resolved Hide resolved

show confusion matrix only when large data is disabled for now

895f595

romanlutz enabled auto-merge (squash) January 12, 2023 05:17

romanlutz disabled auto-merge January 12, 2023 12:10

romanlutz merged commit 1f89e68 into main Jan 12, 2023

romanlutz deleted the t-wangjohn/ConfusionMatrix branch January 12, 2023 12:10

imatiach-msft mentioned this pull request Jan 18, 2023

release raiwidgets and responsibleai 0.24.1 #1914

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Confusion Matrix #1794

Confusion Matrix #1794

t-wangjohn commented Oct 27, 2022 •

edited by romanlutz

Loading

codecov-commenter commented Oct 27, 2022 •

edited

Loading

github-actions bot commented Oct 27, 2022

github-actions bot commented Oct 27, 2022

github-actions bot commented Oct 28, 2022

github-actions bot commented Nov 21, 2022

tongyu-microsoft commented Jan 10, 2023

romanlutz commented Jan 10, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 12, 2023

Confusion Matrix #1794

Confusion Matrix #1794

Conversation

t-wangjohn commented Oct 27, 2022 • edited by romanlutz Loading

Description

Checklist

codecov-commenter commented Oct 27, 2022 • edited Loading

Codecov Report

github-actions bot commented Oct 27, 2022

github-actions bot commented Oct 27, 2022

github-actions bot commented Oct 28, 2022

github-actions bot commented Nov 21, 2022

tongyu-microsoft commented Jan 10, 2023

romanlutz commented Jan 10, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 11, 2023

github-actions bot commented Jan 12, 2023

t-wangjohn commented Oct 27, 2022 •

edited by romanlutz

Loading

codecov-commenter commented Oct 27, 2022 •

edited

Loading