Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Tracker]: Migrate metrics to UWM #1854

Closed
alexcreasy opened this issue Sep 25, 2023 · 6 comments
Closed

[Tracker]: Migrate metrics to UWM #1854

alexcreasy opened this issue Sep 25, 2023 · 6 comments
Assignees
@alexcreasy
Labels
feature/model-serving Model Serving Feature migrated tracker Non-completable ticket; used for tracking work - akin to a Jira Epic

Comments

@alexcreasy
Copy link
Contributor

alexcreasy commented Sep 25, 2023 •
edited
Loading

Description

The backend metrics stack is moving to UWM, we need to migrate the prometheus calls in the dashboard to fetch these from UWM for both performance and bias metrics. This will also allow users who do not have the cluster-admin role to view the metrics.

Target Branch

f/mserving-metrics

Requirements

  • P0: Performance metrics should be fetched from UWM.
  • P0: Performance metrics should be viewable by a project admin, without cluster-admin role.
  • P0: Bias metrics should be fetched from UWM.
  • P0: Bias metrics should be viewable by a project admin, without cluster-admin role.

Itemized UX Issues

N/A

Itemized Dev Issues

Related artifacts

No response
@alexcreasy alexcreasy added the tracker Non-completable ticket; used for tracking work - akin to a Jira Epic label Sep 25, 2023
@alexcreasy alexcreasy moved this from Untriaged to Dev In progress in ODH Dashboard Planning Sep 25, 2023
@alexcreasy alexcreasy moved this to Dashboard in ODH Feature Tracking Sep 25, 2023
@alexcreasy alexcreasy self-assigned this Sep 25, 2023
This was referenced Sep 25, 2023
Closed
Closed
Closed
@christianvogt christianvogt added the feature/model-serving Model Serving Feature label Sep 25, 2023
@andrewballantyne andrewballantyne moved this from Dev In progress to Trackers in ODH Dashboard Planning Sep 26, 2023
@alexcreasy
Copy link
Contributor Author

alexcreasy commented Oct 3, 2023 •
edited
Loading

  • We're currently blocked due to haproxy metric queries not working on UWM - these are used for performance metrics on the dashboard.
  • I've raised a WIP PR for the dashboard that demonstrates TrustyAI bias metrics working along with model server performanmce metrics, however the model performance metrics don't work due to the above issue.
  • Vedant is currently working on a solution for the missing metrics that involves using native modelmesh queries - I'm awaiting more details / estimates - which could lead to more issues being added to this Tracker, I'm not sure - I'll update when I have that info

@alexcreasy
Copy link
Contributor Author

Update: We're no longer blocked waiting for the modelmesh updates for our metric queries. I've finalised a single PR that closes all issues of this tracker[1] - it is now awaiting review before merging to the feature branch.

@alexcreasy
Copy link
Contributor Author

Update: The fix is still awaiting review. I'm working with @pnaik1 & @ppadti to get their clusters up to speed so they can perform the code review before this is merged.

@alexcreasy alexcreasy mentioned this issue Oct 27, 2023
Closed
@dgutride dgutride moved this to In Progress in Internal tracking Oct 27, 2023
@alexcreasy
Copy link
Contributor Author

All issues are complete and this tracker is ready to be moved to incubation when Explainability phase 1 and Performance metrics are moved over.

@dgutride
Copy link
Contributor

dgutride commented Dec 1, 2023

Moving remainder incubation tracking to jira: https://issues.redhat.com/browse/RHOAIENG-241

@dgutride dgutride closed this as completed Dec 1, 2023
@github-project-automation github-project-automation bot moved this from In Progress to Done in Internal tracking Dec 1, 2023
@github-project-automation github-project-automation bot moved this from Trackers to Done in ODH Dashboard Planning Dec 1, 2023
@dgutride dgutride moved this from Done to Migrated in ODH Dashboard Planning Dec 1, 2023
@adnankhan666
Copy link

https://issues.redhat.com/browse/RHOAIENG-1785

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alexcreasy alexcreasy

Labels
feature/model-serving Model Serving Feature migrated tracker Non-completable ticket; used for tracking work - akin to a Jira Epic
Projects
Internal tracking
Status: Done
ODH Feature Tracking
Status: Dashboard
ODH Dashboard Planning
Archived in project
Milestone
No milestone
Development

No branches or pull requests
5 participants
@alexcreasy @dgutride @andrewballantyne @christianvogt @adnankhan666