How to use Trainer with Vision Transformer
What changes should be made for using Trainer with the Vision Transformer,  are the keys expected by the trainer from datasetinput_ids,attention_mask, andlabels?class OCRDataset(torch.utils.data.Dataset):
    def __init__(self, texts, tokenizer, transforms = None):
        self.texts = texts
        self.tokenizer = tokenizer
        self.transforms = transforms

    def __getitem__(self, idx):
        data = generate_sample(self.texts[idx])
        if data:
          img, label = data
          img = torch.from_numpy(img)
          tokens = tokenizer(label, padding='max_length')
        if self.transforms:
          img = self.transforms(img)
        batch = {}
        batch['labels'] = tokens
        batch['input_ids'] = img
        return batch

transform= transforms.Compose([transforms.Normalize((0.5,), (0.5,))])
train_dataset = OCRDataset(jp_list, tokenizer, transform)
.....
.....

trainer.train()This code throws the following errorValueError: could not determine the shape of object type ‘BatchEncoding’
2021-10-18T07:29:20Z
Hi,I do have a demo notebook on using the Trainer for fine-tuning the Vision Transformer here:Transformers-Tutorials/Fine_tuning_the_Vision_Transformer_on_CIFAR_10_with_the_🤗_Trainer.ipynb at master · NielsRogge/Transformers-Tutorials · GitHub.ViT doesn’t expectinput_idsandattention_maskas input, butpixel_valuesinstead. Note that we will add support forattention_maskin the future.
2021-10-19T17:31:25Z
Thanks, this worked for me. Can you tell me how to load the model from a checkpoint after training? In your example, the model trained is an object of theViTForImageClassificationclass, sofrom_pretrained()cannot be used.
2021-10-19T19:06:23Z
from_pretrainedcan be used on any model, so also onViTForImageClassification