Config settings for Building a custom decoder using Pretrained conformer CTC Hindi model as Encoder
Hi, I am trying to use pretrained conformer-ctc-medium Hindi model as encoder, and I would like to add conformer-transducer as decoder initialized with random weights.

I tried following things, but none of them is working:
1. I have used **conformer_transducer_bpe.yaml** thinking that the decoder config parameters would be loaded from the config & "RNNTDecoder" would be added. 
2. I have tried initializing in the python code, but that is also not working:
conf_ctc_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained(model_name="stt_hi_conformer_ctc_medium")
conf_ctc_model.cfg.decoder._target_='nemo.collections.asr.modules.RNNTDecoder' 
3. I have tried adding decoder in python code as 
RNN_decoder  = dict(
_target_ = 'nemo.collections.asr.modules.RNNTDecoder',
normalization_mode = 'null', 
random_state_sampling = 'false', 
blank_as_pad = 'true', 

prednet = dict(
    pred_hidden = 640, 
    pred_rnn_layers = 1,
    t_max= 'null',
    dropout= 0.2
)
)
params['model']['decoder'] = RNN_decoder 

4. In addition to RNN_decoder, tried initializing joint, decoding, loss, variational_noise config parameters. But did not help.
5. Tried change_vocabulary before & after config parameters loading

But, In all cases  decoder was being displayed as ConvASRDecoder, loss = CTCloss, encoder= ConformerEncoder, both during training & in hparams.yaml file.

Please suggest how to use pretrained conformer-ctc with a custom decoder. This decoder can be a conformer/transformer.
or
Can I also try using encoder decoder based "Quartznet" hindi pretrained model to accomplish the same.

Or Can I use pyCTCdecoder for decoding? 
Thanks
