How to improve ASR speed, Tested on Nvidia T4, L4, A100 give me same speed
Can you check if your are loading the model every time ?  This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. This issue was closed because it has been inactive for 7 days since being marked as stale.
Hello, 

I want to know how to speed up NEMO.

Originally, I ran test with **[stt_zh_citrinet_1024_gamma_0_25](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_zh_citrinet_1024_gamma_0_25)** model with 30 seconds audio clip on Nvidia T4,
It give me around 1 - 2 seconds. It too slow for our requirements.

Then, I ran test on both Nvidia L4 and A100 which is more powerful than T4, 
But it still give me same speed (1-2 seconds), 

So, How to improve ASR speed?  Maybe I did something wrong. Please advise

PS. I'm sure CUDA is enabled, Using CPU give me more than 10 seconds

python = 3.10
nemo_toolkit = 1.21
cuda = 12.1
debian = 11