Very poor WER for Conformer_CTC_large model in streaming mode
An offline model is not supposed to be used with cache-aware scripts, specially with small chunk sizes. You need to use cache-aware models like:
https://huggingface.co/nvidia/stt_en_fastconformer_hybrid_large_streaming_multi

Riva does not support cache-aware models publicly yet as far as I know. You may use nemo for cache-aware until they are fully supported in Riva in the meantime.

If for some reasons, you need to use offline models for streaming in nemo, you need to use this script for buffered streaming. Riva is supporting buffered streaming which is a streaming approach for offline-trained models:

https://github.com/NVIDIA/NeMo/blob/main/examples/asr/asr_chunked_inference/ctc/speech_to_text_buffered_infer_ctc.py

Just keep that in mind that in buffered streaming (the one Riva uses), chunk size is not comparable to cache-aware models. They add some eight and left contexts to the chunk which increases the latency, usually buffer size is 4s and the 800ms chunk is the middle of the buffer. This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. This issue was closed because it has been inactive for 7 days since being marked as stale.
Hello. I am testing the [STT En Conformer-CTC Large](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_en_conformer_ctc_large/files) model in streaming mode. To do this, I use the [speech_to_text_cache_aware_streaming_infer.py](https://github.com/NVIDIA/NeMo/blob/main/examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py) script and the [librispeech-test-other](https://www.openslr.org/12) dataset. With the default `chunk_size` value of 4000 ms, everything works quite well. The WER is 8.67%. However, when I reduce the `chunk_size` to 800 ms, I get very poor results. The WER increases to 35.6%.
I understand that this model was trained with full context in offline mode. And the accuracy would not be very good with small chunks. But I also tested this model in streaming mode (`chunk_size` = 800ms) with RIVA. I converted the nemo model to riva with [nemo2riva](https://github.com/nvidia-riva/nemo2riva) and built a pipeline. In this case, the WER was 7.9%. I understand that the RIVA service contains certain techniques and optimizations for working with the streaming mode. But I wonder why there is such a difference in WER when using the same model and the same chunk size. 
Can anyone explain this behavior? Maybe RIVA reconfigures the model in some way or something else that I might have missed?