mms-1b-toigen-female-model

This model is a fine-tuned version of facebook/mms-1b-all on the TOIGEN - TOI dataset. It achieves the following results on the evaluation set:

• Loss: 0.2197
• Wer: 0.3590

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.8474	0.4016	100	3.6385	0.9952
2.2373	0.8032	200	0.5181	0.6290
0.6093	1.2048	300	0.3649	0.5187
0.4813	1.6064	400	0.3151	0.4909
0.3843	2.0080	500	0.2994	0.4567
0.3805	2.4096	600	0.2818	0.4423
0.3591	2.8112	700	0.2811	0.4402
0.3164	3.2129	800	0.2688	0.4105
0.3351	3.6145	900	0.2569	0.4060
0.3442	4.0161	1000	0.2639	0.4
0.2996	4.4177	1100	0.2581	0.4085
0.3196	4.8193	1200	0.2444	0.3863
0.3147	5.2209	1300	0.2450	0.3863
0.2936	5.6225	1400	0.2414	0.3855
0.2894	6.0241	1500	0.2363	0.3722
0.2772	6.4257	1600	0.2452	0.3843
0.2565	6.8273	1700	0.2357	0.3759
0.3033	7.2289	1800	0.2358	0.3787
0.2651	7.6305	1900	0.2349	0.3634
0.2912	8.0321	2000	0.2305	0.3658
0.2554	8.4337	2100	0.2303	0.3678
0.2535	8.8353	2200	0.2269	0.3553
0.2368	9.2369	2300	0.2288	0.3610
0.2491	9.6386	2400	0.2244	0.3686
0.2678	10.0402	2500	0.2289	0.3561
0.2247	10.4418	2600	0.2234	0.3614
0.2773	10.8434	2700	0.2248	0.3541
0.2162	11.2450	2800	0.2197	0.3590
0.2526	11.6466	2900	0.2208	0.3521
0.2521	12.0482	3000	0.2230	0.3702
0.2218	12.4498	3100	0.2208	0.3529
0.2419	12.8514	3200	0.2204	0.3517

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0