📊 Model-wise Classification Count Tracking

[Jai-76]

Implement a feature to track and display classification counts for each model in the system. This will help in analyzing performance trends, monitoring model usage, and identifying imbalances in predictions.
Goals
Maintain a running count of predictions per model.
Categorize counts based on classification labels.
Provide a real-time or periodic update mechanism for counts.

[HARSHITA005-GARG]

@eccentriccoder01 can I work on this issue please?

[eccentriccoder01]

@HARSHITA005-GARG @Jai-76 there are already various counts and tracking info incorporated into the app. Could you elaborate specifically on what changes/improvements you'd like to make, so that I can accordingly assign the better idea and the level?

[Jai-76]

The proposal is to implement a feature that tracks and displays classification counts for each model within the system. This enhancement aims to facilitate the analysis of performance trends, monitor model usage, and identify any imbalances in predictions.

Maintain a Running Count: Keep an accurate and up-to-date count of predictions made by each model.
Categorize Counts: Organize counts based on classification labels to provide insights into model performance across different categories.
Real-time Updates: Implement a mechanism for real-time or periodic updates of counts to ensure that users have access to the latest data.

[HARSHITA005-GARG]

Capture Predictions – After each inference, log model_name + predicted_label(s).

Store Counts – Use an in-memory counter (e.g., dict, Redis) for fast updates.

Increment per Label – For multi-label outputs, increment all relevant labels.

Persist Data – Periodically save counts to a database for history.

Expose API – Create an endpoint (e.g., /metrics/model_counts) to fetch counts.

Update Mechanism – Choose real-time (immediate updates) or periodic batch updates.

Visualize – Use Grafana, Streamlit, or dashboard to display per-model counts & trends.

Monitor Imbalances – Add alerts for unusual shifts in label distribution.

[eccentriccoder01]

Alright @HARSHITA005-GARG, your suggestions are more suited to the purpose of this issue. Just make sure to:

• Stay updated by fetching changes from the upstream in your local repo, as multiple people are/will be constantly updating the main.
• Include a working deployment link of your PR and relevant screenshots.
• Reach out in case of any doubts...

Assigning this issue to you, 2 days for this. All the best!

@Jai-76 kindly respond to my questions in your other PRs and Issues. Thanks.

[Jai-76]

I will make wait

[Kavlin-Kaur]

@eccentriccoder01 can u please assign this to me!!!
Thank you so much

[eccentriccoder01]

Sure @Kavlin-Kaur!