Disable T5 Test
Or, a better solution would be to disable T5 as a choice from https://github.com/NVIDIA/NeMo/blob/693c556f7bcffaf78e698f69b36331d872abf533/nemo/collections/nlp/modules/common/lm_utils.py#L44

And in this CI test config use "bert-base-uncased"? @ekmb  what do you think? > Why would we disable the test? Let’s just put an upper bound on HF transformers lib version and add properly fixing this model to our to-do list

T5 is experimental and no support is guaranteed.
It doesn't look like anyone is planning on fixing this model/test.
Pinning python library versions is not a fix.

I am ok with pinning libraries if someone plans on fixing the test, but it looks like no one is available. Since it won't be fixed, the easiest path to unblock existing PRs is just to remove the test. > Or, a better solution would be to disable T5 as a choice from
> 
> https://github.com/NVIDIA/NeMo/blob/693c556f7bcffaf78e698f69b36331d872abf533/nemo/collections/nlp/modules/common/lm_utils.py#L44
> 
> And in this CI test config use "bert-base-uncased"? @ekmb what do you think?

Does this test improve our CI coverage? Our CI is reaching >20 mins now which is frustratingly slow. > > Why would we disable the test? Let’s just put an upper bound on HF transformers lib version and add properly fixing this model to our to-do list
> 
> T5 is experimental and no support is guaranteed.
> It doesn't look like anyone is planning on fixing this model/test.
> Pinning python library versions is not a fix.
> 
> I am ok with pinning libraries if someone plans on fixing the test, but it looks like no one is available. Since it won't be fixed, the easiest path to unblock existing PRs is just to remove the test.

Yeah, let's remove T5 as a choice from ```get_lm_model``` function (e.g. raise exception if T5 is requested) and not use that model (and use bert-base) for this CI test. Then we won't need to pin library and won't have broken example. > > > Why would we disable the test? Let’s just put an upper bound on HF transformers lib version and add properly fixing this model to our to-do list
> > 
> > 
> > T5 is experimental and no support is guaranteed.
> > It doesn't look like anyone is planning on fixing this model/test.
> > Pinning python library versions is not a fix.
> > I am ok with pinning libraries if someone plans on fixing the test, but it looks like no one is available. Since it won't be fixed, the easiest path to unblock existing PRs is just to remove the test.
> 
> Yeah, let's remove T5 as a choice from `get_lm_model` function (e.g. raise exception if T5 is requested) and not use that model (and use bert-base) for this CI test. Then we won't need to pin library and won't have broken example.

Let's remove this test, bert-base is tested in other places. I'll add refactoring of 'get_lm_model' on my to-do.