Subject: Potential Overfitting Issue and Loss Representation in Inference

[z-yf17]

Hello everyone,

I am currently working on a project where I encountered some issues during training and inference. Specifically, I have noticed that while the training loss decreases, the success rate during inference first increases and then decreases. This has led me to question whether overfitting might be occurring in my model.

Here are my main concerns:

Could this pattern, where the inference success rate first improves and then declines despite a continuously decreasing training loss, indicate overfitting?
Can loss effectively represent the performance of the model during inference, or are there other metrics that might be more indicative?
I would appreciate any insights or suggestions on how to address these issues.

Thank you!