Is that possible to change the input shape in Trainer?
Can we use different input shape for transformer model like DistilBERT?I am going to build a language model with MLM, over DistilBERT model. As we know the input shape of the model is (* , 512), so how we can use (*, 64) as an input? (That is due to not that long sequences)I useTrainerfor training the model for domain adaption and DistilBERT as a model.