swiftformer-xs-OT

This model is a fine-tuned version of MBZUAI/swiftformer-xs on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.4956
• Accuracy: 0.8548

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0015
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.89	4	1.3804	0.5161
No log	2.0	9	1.2554	0.5323
1.3469	2.89	13	0.9725	0.6613
1.3469	4.0	18	0.7086	0.7581
0.9831	4.89	22	0.8856	0.7258
0.9831	6.0	27	0.7724	0.7581
0.7441	6.89	31	0.8190	0.7258
0.7441	8.0	36	0.6897	0.7742
0.6939	8.89	40	0.6599	0.7258
0.6939	10.0	45	0.6288	0.7742
0.6939	10.89	49	0.6333	0.7581
0.5861	12.0	54	0.6206	0.7742
0.5861	12.89	58	0.5263	0.7903
0.5018	14.0	63	0.5836	0.8065
0.5018	14.89	67	0.6125	0.7419
0.4642	16.0	72	0.5431	0.8065
0.4642	16.89	76	0.5893	0.8387
0.4064	18.0	81	0.4997	0.8065
0.4064	18.89	85	0.5474	0.7742
0.4275	20.0	90	0.6748	0.7903
0.4275	20.89	94	0.6369	0.7581
0.4275	22.0	99	0.5610	0.7742
0.373	22.89	103	0.5260	0.7903
0.373	24.0	108	0.5416	0.8387
0.2931	24.89	112	0.5146	0.8387
0.2931	26.0	117	0.5180	0.7742
0.3135	26.89	121	0.5169	0.8226
0.3135	28.0	126	0.5491	0.8387
0.2342	28.89	130	0.5385	0.8387
0.2342	30.0	135	0.5456	0.8387
0.2342	30.89	139	0.4956	0.8548
0.2411	32.0	144	0.5254	0.8226
0.2411	32.89	148	0.5533	0.8387
0.2135	34.0	153	0.5613	0.8387
0.2135	34.89	157	0.5748	0.8226
0.1904	35.56	160	0.5844	0.8387

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0