Unable to deploy nemo model to riva pipeline (after conversion to riva)
**Describe the bug**

Unable to deploy a fine-tuned Punctuation and Capitalization model to Riva pipeline

**Steps/Code to reproduce bug**

1. Fine tune a nemo punctuation and capitalization model. 
2. Convert it to rive using this guide: https://docs.nvidia.com/deeplearning/riva/user-guide/docs/model-overview.html
3. Deploy it using nvcr.io/nvidia/riva/riva-speech:2.7.0 and it is giving an empty response. 

Deploy command
NV_GPU=1 nvidia-docker run -ti --name riva_source -p 8008:8008 -p 50051:50051 -p 5800:8000 -p 58001:8001 -v <localtion_models>:/data/models -e MODEL_REPOSITORY="--model-repository /data/models" -e INTERIM_RESULTS="--interim-results"  -e CUSTOM_ITN="--custom-itn" --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/riva/riva-speech:2.4.0-server tritonserver --log-verbose=0 --strict-model-config=true --model-repository /data/models --cuda-memory-pool-byte-size=0:1000000000 --exit-on-error=false

This was working while using riva 1.10-beta server

A  helpful guide on on how to craft a minimal bug report  http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports. 

My ourput logs have the following. Please note the output contents are empty
model_name: "riva-punctuation-en-US"
id: "1"
inputs {
  name: "PIPELINE_INPUT"
  datatype: "BYTES"
  shape: 1
  shape: 1
}
outputs {
  name: "PIPELINE_OUTPUT"
}
raw_input_contents: "\372\000\000\000in a new password i can take it from here alrighty is there anything else i can help you with them thank you very much ms rostow it was my pleasure to assist you thank you for using optum r x and if you chose our survey it will be next okay thank you"

model_name: "riva-punctuation-en-US"
model_version: "1"
id: "1"
outputs {
  name: "PIPELINE_OUTPUT"
  datatype: "BYTES"
  shape: 1
  shape: 1
}
raw_output_contents: "\000\000\000\000"

[[b'']]

**Expected behavior**

Deploy the model correctly and output content is a punctuated input string

**Environment overview (please complete the following information)**

 - Environment location: nvcr.io/nvidia/riva/riva-speech:2.4.0-server
 - Method of NeMo install: install from source
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
nvcr.io/nvidia/riva/riva-speech:2.4.0-server
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: GPU model
