wav2vec CTC val_wer is always 1
**Describe the bug**

Hi, I am having issues with training wav2vec models. It seems like the model starts training, there are no warnings or errors, everything runs. However, after every epoch val_wer is equal to 1.0. For [wav2vec_pretrain.yaml](https://github.com/NVIDIA/NeMo/blob/main/examples/asr/conf/ssl/wav2vec/wav2vec_pretrain.yaml) I am not sure where to start as there are no tutorials or examples. So I have been using [wav2vecCTC.yaml](https://github.com/NVIDIA/NeMo/blob/main/examples/asr/conf/wav2vec_ctc/wav2vecCTC.yaml) and the example script for training CTC models.

**Steps/Code to reproduce bug**

Below is the exact used command, used datasets work perfectly with other NeMo ASR models. It is the same command that I use for all CTC models.

```
python /workspace/nemo/examples/asr/asr_ctc/speech_to_text_ctc_bpe.py \\
  --config-path=/workspace/nemo/examples/asr/conf/wav2vec_ctc \\
  --config-name=wav2vecCTC \\
  exp_manager.name=wav2vec} \\
  +exp_manager.exp_dir=/experiments \\
  exp_manager.resume_if_exists=true \\
  exp_manager.resume_ignore_no_checkpoint=true \\
  +exp_manager.checkpoint_callback_params.save_best_model=true \\
  model.train_ds.manifest_filepath=/data/train.nemo \\
  model.train_ds.batch_size=8 \\
  model.train_ds.num_workers=4 \\
  model.train_ds.max_duration=17 \\
  +model.train_ds.min_duration=1 \\
  +model.train_ds.parser=base \\
  model.validation_ds.manifest_filepath=/data/validation.nemo \\
  model.validation_ds.batch_size=8 \\
  model.validation_ds.num_workers=4 \\
  +model.validation_ds.parser=base \\
  model.tokenizer.dir=/data/out/tokenizer_spe_unigram_v128 \\
  model.tokenizer.type=bpe \\
  trainer.precision=bf16 \\
  trainer.devices=-1
```

**Expected behavior**

I expect val_wer to drop with training and not remain at its maximum value (1.0).

**Environment overview (please complete the following information)**

 - Official latest NeMo container where the NeMo repository is updated to the current tip of the master branch.

**Environment details**

 - Official latest NeMo container where the NeMo repository is updated to the current tip of the master branch.

**Additional context**

Running on 4 RTX A6000 GPUs.
