Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vision model overview metrics #1688

Merged
merged 2 commits into from
Sep 11, 2022
Merged

Conversation

jamesbchao
Copy link
Contributor

@jamesbchao jamesbchao commented Sep 6, 2022

Calculates Accuracy, Precision, Recall, and F1 scores for vision data in the Model Overview tab. Implements calculation of macro Precision, Recall, and F1 scores for multiclass classification scenarios. Compatible with other metadata, but dependent on Vision dashboard cohorts and Model assessment vision.

Update: micro metrics
Screenshot (80)

Screenshot (75)
Screenshot (76)

Description

Checklist

  • I have added screenshots above for all UI changes.
  • I have added e2e tests for all UI changes.
  • Documentation was updated if it was needed.

@codecov-commenter
Copy link

codecov-commenter commented Sep 6, 2022

Codecov Report

Merging #1688 (aff5aaf) into main (ded68a6) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1688   +/-   ##
=======================================
  Coverage   88.91%   88.91%           
=======================================
  Files         105      105           
  Lines        5559     5559           
=======================================
  Hits         4943     4943           
  Misses        616      616           
Flag Coverage Δ
unittests 88.91% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@jamesbchao jamesbchao force-pushed the jbchao/vision-dashboard-metrics branch 3 times, most recently from ab7ebea to a64c90b Compare September 8, 2022 01:55
3 similar comments
@jamesbchao jamesbchao force-pushed the jbchao/vision-dashboard-metrics branch from 7c0ed8e to 0856a08 Compare September 8, 2022 21:27
@jamesbchao jamesbchao force-pushed the jbchao/vision-dashboard-metrics branch 2 times, most recently from 669f519 to 6a5651a Compare September 9, 2022 00:43
1 similar comment
@imatiach-msft
Copy link
Contributor

@jamesbchao it looks like the tabular data UI tests are failing:

should have 'Model overview' component in the initial state:

AssertionError: Timed out retrying: Too many elements found. Found '5', expected '2'.

It looks like the PR is affecting the tabular data case for model overview metrics, and this is causing the tabular data ui tests to fail.
I'm guessing it's due to the new metrics added.

@jamesbchao jamesbchao force-pushed the jbchao/vision-dashboard-metrics branch from 6a5651a to 0a1f174 Compare September 10, 2022 00:17
@imatiach-msft imatiach-msft force-pushed the jbchao/vision-dashboard-metrics branch from a7cbf6e to 0761b03 Compare September 11, 2022 03:35
@imatiach-msft imatiach-msft force-pushed the jbchao/vision-dashboard-metrics branch from 0761b03 to aff5aaf Compare September 11, 2022 03:40
@imatiach-msft imatiach-msft merged commit b9bd5b7 into main Sep 11, 2022
@imatiach-msft imatiach-msft deleted the jbchao/vision-dashboard-metrics branch September 11, 2022 05:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants