How to reduce the vocabulary for a QuartzNet model?
For a simple script like this:
```python

#!/usr/bin/env python
import nemo.collections.asr as nemo_asr
import ctc_decoders

import numpy as np
import librosa

import os, sys, pdb

## Instantiate model
DetailsDict = nemo_asr.models.EncDecCTCModel.list_available_models()
asr_model = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name='QuartzNet15x5Base-En', strict=False)

## Test audio clip
AUDIO_FILENAME = 'test.wav'
signal, sample_rate = librosa.load(AUDIO_FILENAME, sr=None)

## Convert our audio sample to text
files = [AUDIO_FILENAME]
transcript = asr_model.transcribe(paths2audio_files=files)[0]; print(f'Transcript: "{transcript}"')
```

It performs well
However if i change the problem of just predicting fixed number of words, for example `yes, no`
How should i augment the language model part of it ?
