My input sentence is very long(more than 512). What should I do when I want to fintune model about classify?Thanks
I know I can intercept the first 512 lengths.But I don’t want to do this. This task can understand the classification of  texts.AutoModelForSequenceClassification can be directly used for classification, but the question is whether the input can be some 512 length sentences, and use pooler layer to classify ?Or what should I do? Thanks
2021-09-02T13:03:46Z
Hey@ccfeidaoyou might want to try one of the dedicated models like LongFormer or BigBird which have a longer context size of around 4,096 tokens. Seethis threadfor more details
2021-09-02T13:39:35Z
Thanks ，I will try
2021-09-03T04:49:20Z
I want to user BigBird to try my task, but have a newproblem, I am very Confusion.