mms-1b-bigcgen-female-25hrs-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.5276

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.463	0.0602	100	inf	1.0084
6.1642	0.1204	200	inf	0.9995
5.3648	0.1806	300	inf	1.0613
3.2037	0.2407	400	inf	0.6218
1.9789	0.3009	500	inf	0.6027
1.7417	0.3611	600	inf	0.5821
1.7214	0.4213	700	inf	0.5575
1.6026	0.4815	800	inf	0.5557
1.652	0.5417	900	inf	0.5507
1.6349	0.6019	1000	inf	0.5478
1.4745	0.6621	1100	inf	0.5373
1.6209	0.7222	1200	inf	0.5473
1.5453	0.7824	1300	inf	0.5287
1.5794	0.8426	1400	inf	0.5326
1.3721	0.9028	1500	inf	0.5251
1.5622	0.9630	1600	inf	0.5269
1.3979	1.0229	1700	inf	0.5351
1.4231	1.0831	1800	inf	0.5301
1.3792	1.1432	1900	inf	0.5305
1.4868	1.2034	2000	inf	0.5401
1.4477	1.2636	2100	inf	0.5296
1.4482	1.3238	2200	inf	0.5262
1.3896	1.3840	2300	inf	0.5226
1.3938	1.4442	2400	inf	0.5246
1.3464	1.5044	2500	inf	0.5274

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0