Serious issue regarding channel dimensions with respect to configuration during training a vision transformer
I am finetuning ‘google/vit-base-patch16-224-in21k’ for multi class image classification task.I have preprocessed my images with AutoImageProcessor. When I start finetuning my model I am getting an error.ValueError: Make sure that the channel dimension of the pixel values match with the one set in the configuration. Expected 3 but got 224I am in a very frustrated situation and no idea how to proceed.After preprocessing, my image size matches with the one in configuration.>>> ISIC_holdout['train'][0]['pixel_values'].shape
(224, 224, 3)This is Configuration of my checkpoint.ViTConfig {“_name_or_path”: “google/vit-base-patch16-224-in21k”,“architectures”: [“ViTModel”],“attention_probs_dropout_prob”: 0.0,“encoder_stride”: 16,“hidden_act”: “gelu”,“hidden_dropout_prob”: 0.0,“hidden_size”: 768,“image_size”: 224,“initializer_range”: 0.02,“intermediate_size”: 3072,“layer_norm_eps”: 1e-12,“model_type”: “vit”,“num_attention_heads”: 12,“num_channels”: 3,“num_hidden_layers”: 12,“patch_size”: 16,“qkv_bias”: true,“transformers_version”: “4.44.0”}
2024-08-26T09:13:36Z
hi@vigneshwar472I think you missed the preprocess part. Image shape should be (3,224,224) rather than (224,224,3).I hope this one will help you:huggingface.coImage classificationWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2024-08-26T13:06:51Z
Yes indeed, thanks for the reply@mahmutc, PyTorch models use the so-called “channels-first” format by default.