You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When the SpeechTestDataCI workflow runs after a test data update, it grabs the latest Custom Speech model to test. However, the list of Custom Speech models includes models that are Succeeded and also models that are Processing.
This results in undesirable behavior when the SpeechTestDataCI workflow gets a model that is Processing. The test will still run, but it will wait the full 40+ min. for that model to finish training before doing so.
This also means that the whole point of the SpeechTestDataCI workflow - the fair comparison - is not happening when testing and training data are updated at the same time. The SpeechTestDataCI workflow will test the new model instead of the benchmark or baseline model.
Solution
The fix for this should be pretty easy. In the output of the azurespeechcli for the speech model list command, we already filter based on the model kind. We can also now filter to only keep models that are Succeeded which will solve this problem.
The text was updated successfully, but these errors were encountered:
When the SpeechTestDataCI workflow runs after a test data update, it grabs the latest Custom Speech model to test. However, the list of Custom Speech models includes models that are
Succeeded
and also models that areProcessing
.This results in undesirable behavior when the SpeechTestDataCI workflow gets a model that is
Processing
. The test will still run, but it will wait the full 40+ min. for that model to finish training before doing so.This also means that the whole point of the SpeechTestDataCI workflow - the fair comparison - is not happening when testing and training data are updated at the same time. The SpeechTestDataCI workflow will test the new model instead of the benchmark or baseline model.
Solution
The fix for this should be pretty easy. In the output of the azurespeechcli for the
speech model list
command, we already filter based on the model kind. We can also now filter to only keep models that areSucceeded
which will solve this problem.The text was updated successfully, but these errors were encountered: