distilbert-classn-LAlg-multihead-context-width-3

This model is a fine-tuned version of dslim/distilbert-NER on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9847
• Accuracy: 0.7302
• F1: 0.7381
• Precision: 0.7640
• Recall: 0.7302

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 25
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
2.4887	1.3514	50	2.4407	0.0952	0.0480	0.0570	0.0952
2.439	2.7027	100	2.4237	0.0794	0.0409	0.0351	0.0794
2.4316	4.0541	150	2.4033	0.0794	0.0583	0.0495	0.0794
2.373	5.4054	200	2.3811	0.0714	0.0533	0.0442	0.0714
2.3379	6.7568	250	2.3562	0.1190	0.1166	0.1993	0.1190
2.2463	8.1081	300	2.2906	0.1667	0.1557	0.1742	0.1667
2.0981	9.4595	350	2.1795	0.2460	0.2334	0.2607	0.2460
1.8632	10.8108	400	1.9812	0.3333	0.3211	0.3410	0.3333
1.5487	12.1622	450	1.7584	0.4286	0.4144	0.4633	0.4286
1.111	13.5135	500	1.4453	0.5635	0.5531	0.5696	0.5635
0.8052	14.8649	550	1.2854	0.6429	0.6440	0.6518	0.6429
0.4797	16.2162	600	1.1147	0.7063	0.7125	0.7317	0.7063
0.3051	17.5676	650	1.0389	0.7063	0.7156	0.7413	0.7063
0.1851	18.9189	700	1.0104	0.7222	0.7307	0.7536	0.7222
0.1166	20.2703	750	0.9889	0.7302	0.7381	0.7638	0.7302
0.0807	21.6216	800	0.9977	0.7302	0.7386	0.7669	0.7302
0.068	22.9730	850	0.9902	0.7381	0.7453	0.7710	0.7381
0.0509	24.3243	900	0.9847	0.7302	0.7381	0.7640	0.7302

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.1
• Tokenizers 0.21.0