How to deploy NeMo ASR models at scale? #10248
asifjahmed
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello, I have built a streaming ASR service using NeMo RNNT and CTC models. I am looking for suggestions on how to deploy this in a production environment. I want to implement continuous batching and utilize the GPU to the fullest to support a large number of concurrent requests. Any guidance on this is much appreciated!
Beta Was this translation helpful? Give feedback.
All reactions