Tokenizer truncation
I’m trying to run sequence classification with a trained Distilibert but I can’t get truncation to work properly and I keep gettingRuntimeError: The size of tensor a (N) must match the size of tensor b (512) at non-singleton dimension 1.I can work around it by manually truncating all the documents I pass into the classifier, but that’s really not ideal.Here is my setup for the pipeline:model_dir = "./classifier_52522_3"
tokenizer = AutoTokenizer.from_pretrained(
    model_dir, model_max_length=512, max_length=512, padding="max_length", truncation=True
)
config = DistilBertConfig.from_pretrained(model_dir)
model = DistilBertForSequenceClassification(config)

pipe = TextClassificationPipeline(
    model=model, 
    tokenizer=tokenizer,
    return_all_scores=True
)I have tried adding the truncation params directly to the savedtokenizer_config.jsonfile too but no dice.Thanks!
2022-06-14T05:34:52Z
Can you provide a bit more information about this, notably the call stack (or its most relevant subset)? I cannot provide help, but would like to compare with an issue I have, which I think is similar.From scouting this forum there are probably quite a few beginners that get stuck on it as well, trying to adapt example fine-tuning workflows to their own data.I get a similar error message when calling thetrainmethod on aTrainer. From a beginner’s perspective this is frustrating as the valueNof the tensor a is not coming from an obvious place.I can see in debug mode in my case that this tensor dimension of size N arises frominputs_embeds = self.word_embeddings(input_ids)somewhere intransformers.models.deberta_v2.modeling_deberta_v2.DebertaV2Embeddings, but this is hardly comforting for the newbie (me).I am not using theTextClassificationPipelinelike you, but I gather this is probably at an equivalent place (training) that the exception occurs?I’ve just publisheda blog post, Lithology classification using Hugging Face, part 2with much more details about this.