[Bug]: No data displayed on bias graphs

[alexcreasy]

Is there an existing issue for this?

• I have searched the existing issues

Deploy type

OpenDataHub core version (eg. v1.6.0)

Version

v1.9.0

Current Behavior

No data is shown on bias charts when data should be available

Expected Behavior

Charts of the data should be shown

Steps To Reproduce

1. Deploy a model with some training data for trustyai
2. Create a metric to observe
3. Send real world data to the model
4. Observe charts on bias tab of model - empty state is shown on charts

Workaround (if any)

No response

What browsers are you seeing the problem on?

No response

Anything else

Looking in the console the query retrieves no data from prometheus.

[alexcreasy]

After some initial investigation, it appears the dashboard is searching the Thanos querier for data. I've confirmed with the TrustyAI team that as of TrustyAI 1.9 they're pushing data to UWM and 'odh-model-monitoring'. This is likely the root cause, which will require a dashboard update to fix.

[alexcreasy]

To provide a bit more context as to what has happened. TrustyAI was previous posting metrics to thanos-querier, however they encountered an issue where metrics were only retained for 30 minutes. Whilst we await transition to UWM they moved back to opendatahub-model-monitoring as the data source. Fixing this should be relatively straightforward and provide a temporary solution until the UWM transition.