Punctuation_and_Capitalization model finetuning
I'm following this tutorial to train a punctuation and capitalization model: https://github.com/NVIDIA/NeMo/blob/stable/tutorials/nlp/Punctuation_and_Capitalization.ipynb
and trying to finetune punctuation_en_distilbert model for Spanish in Google Colab.

When I use tatoeba Spanish dataset ~ 300K lines it works, but validation loss does not decrease and is about 0.2 - 0.25.
When I took a larger dataset, 3M lines, then there was not enough memory for gpu and runtime crashed on pretrained_model.setup_training_data() step.
When I took 1M lines, again val loss for 15 epochs was around 0.2 - 0.25, and does not decrease.

Is there any tips what could I do to get good result for finetuned model?