Explainability: Fairness and Bias Metrics (Phase 0) #1001
Assignees
Labels
feature/model-serving
Model Serving Feature
tracker
Non-completable ticket; used for tracking work - akin to a Jira Epic
Comments
alexcreasy
added
the
tracker
|
Not going to be part of the next release -- we'll do more internal iterations on it before we release it. |
|
The tech preview UI for the TrustyAI phase0 April 12th demo is complete and available here: #1093 This will need more iterations: both to improve code quality and UX before releasing. Planning for this will begin after the April 12th demo is delivered. |
andrewballantyne
changed the title
alexcreasy
added a commit
to alexcreasy/odh-dashboard
that referenced
this issue
Apr 18, 2023
) (opendatahub-io#1006) (opendatahub-io#1007) (opendatahub-io#1008) - Initial feature set for TrustyAI related UI functionality - Adds tab based navigation to modelServing screen - Adds a bias metrics tab with charts for visualising SPD and DIR metrics - Enhances prometheus query features for accessing TrustyAI data - Enhacements to MetricsChart component making it more configurable
alexcreasy
added a commit
to alexcreasy/odh-dashboard
that referenced
this issue
Apr 21, 2023
) (opendatahub-io#1006) (opendatahub-io#1007) (opendatahub-io#1008) - Initial feature set for TrustyAI related UI functionality - Adds tab based navigation to modelServing screen - Adds a bias metrics tab with charts for visualising SPD and DIR metrics - Enhances prometheus query features for accessing TrustyAI data - Enhacements to MetricsChart component making it more configurable
Merged
6 tasks
|
PR awaiting final review to close tracker and related issues: #1093 |
openshift-merge-robot
pushed a commit
that referenced
this issue
Apr 25, 2023
* Explainability: Fairness and Bias Metrics (Phase 0) (#1001) (#1006) (#1007) (#1008) - Initial feature set for TrustyAI related UI functionality - Adds tab based navigation to modelServing screen - Adds a bias metrics tab with charts for visualising SPD and DIR metrics - Enhances prometheus query features for accessing TrustyAI data - Enhacements to MetricsChart component making it more configurable * Update key of request name to match trusty backend * Remove unnecessary div and inline style from tooltip * Remove 15 minutes refresh option * Prefer optional prop to type union with undefined * Move function definitions inline * Prefer narrowing over type conversion * Inline tab change handler * Remove toolbar option from ApplicationsPage * Inline domain calculator functions * Move defaultDomainCalculator to utils * Return null instead of undefined * Use threshold label instead of index for key * Add enum for tab keys * Remove magic numbers from domain calculations * Make ResponsePredicate mandatory and add predicate to useQueryRangeResourceData
github-project-automation
bot
moved this from In review
to Done
in ODH Dashboard Planning
openshift-merge-robot
pushed a commit
that referenced
this issue
Jun 30, 2023
* Re-enable Metrics * Fix Graphs & Configure Infrastructure (#1023) * Rework Inference Metrics & Add Runtime Metrics (invalid quieries) * Add stacked line chart functionality * Trustyai demo phase0 (#1093) * Explainability: Fairness and Bias Metrics (Phase 0) (#1001) (#1006) (#1007) (#1008) - Initial feature set for TrustyAI related UI functionality - Adds tab based navigation to modelServing screen - Adds a bias metrics tab with charts for visualising SPD and DIR metrics - Enhances prometheus query features for accessing TrustyAI data - Enhacements to MetricsChart component making it more configurable * Update key of request name to match trusty backend * Remove unnecessary div and inline style from tooltip * Remove 15 minutes refresh option * Prefer optional prop to type union with undefined * Move function definitions inline * Prefer narrowing over type conversion * Inline tab change handler * Remove toolbar option from ApplicationsPage * Inline domain calculator functions * Move defaultDomainCalculator to utils * Return null instead of undefined * Use threshold label instead of index for key * Add enum for tab keys * Remove magic numbers from domain calculations * Make ResponsePredicate mandatory and add predicate to useQueryRangeResourceData * TrustyAI Client (#1318) * Add support for insecure http requests in development mode * Adds low level API client for TrustyAI service * Adds TrustyAI high level API and contexts * Get scheme of TrustyAI route from k8s data * Add model bias configuration table (#1290) * Add model bias configuration table * rebase and remove mock data * Update Trusty AI client to handle API changes (#1336) (#1337) * Add bias metrics configuration modal (#1343) * Add configuration modal * address comments * get rid of some TODOs and refine the route * Multi-metric display on model bias screen (#1273) (#1349) * Enhancements to model bias screen * Display of multiple bias charts simultaneously * Multi-select component, allowing free text, or select-from-list selection of chartst to display * Ability to collapse / expand individual charts * User selectable refresh rates of chart data * Chart selection and open / closed status is persisted to session cache for life of user's browser session Display user defined threshold values on charts (#1163) * Clean up of bias chart logic * Displays thresholds chosen by user, or defaults if none. * Improves domain and threshold calculation based on user values or defaults * Fix metrics submission issue and handle errors (#1378) * Fix metrics submission issue and handle errors * fix lint issue * use error handler on GET functions * Default and restrict threshold, add tooltips, default duplicate name and set feature flag (#1390) * Default and restrict threshold, add tooltips, default duplicate name and set feature flag * fix lint * add tooltips and dropdown descriptions * clear data when closing configuration modal * really solve deleting issue, make empty table view a common component and apply it everywhere * address comments * Minor enhancements to bias chart (#1386) (#1399) * Adds refresh interval options that match openshift observability dashboard * Show first chart from list, if none selected when user first navigates to bias tab * Use search icon instead of plus for nothing selected empty state * Fix error with calculation of 30 days constant * Deleted charts are removed from session storage * Fixes issue with bias charts auto-refreshing with stale data (#1403) (#1404) * Refactor prometheus queries to remove duplication * Fix graph not refreshing issue * Add code review suggestions * Add performance metrics feature flag and refactor runtime server route (#1413) * Add performance metrics feature flag, refactor runtime server route and solve layout issues * revert some style changes * Model serving metrics renaming (#1421) * Adds support for TrustyAI Operator (#1443) * Adds support for TrustyAI Operator (#1276) * Changes from feedback --------- Co-authored-by: Andrew Ballantyne <aballant@redhat.com> Co-authored-by: Andrew Ballantyne <8126518+andrewballantyne@users.noreply.github.com> Co-authored-by: Alex Creasy <acreasy@redhat.com> Co-authored-by: Alex Creasy <arcreasy@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The text was updated successfully, but these errors were encountered: