Deploy ASR STT Streaming model
@vsl9 can we deploy ASR Nemo checkpoints on Triton ?  This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. This issue was closed because it has been inactive for 7 days since being marked as stale.
Hi

Any suggestions on how to deploy and scale the STT En FastConformer Hybrid Transducer-CTC Large Streaming Multi model. We have finetuned this model and need to handle 100 users (to start with) using this model.

We will receive the chunks of audio data through a websocket connection from our application. How do we tackle multiple users? Do we create multiple instances of the model? Do we connect multiple users to same model? We are using the following tutorial for doing the inference through the model. (https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Online_ASR_Microphone_Demo_Cache_Aware_Streaming.ipynb)
This notebook has a transcribe chunk function which takes an audio chunk and processes it. Can we do batch transcription here for multiple users?

Thank You