be802bd9-6b6a-4b00-b0f9-ee6f2ddf222c

This model is a fine-tuned version of Xenova/tiny-random-Phi3ForCausalLM on the None dataset. It achieves the following results on the evaluation set:

• Loss: 10.2854

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.000204
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 50
• training_steps: 500

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0004	1	10.3797
10.3397	0.0181	50	10.3382
10.3335	0.0362	100	10.3335
10.3097	0.0543	150	10.3112
10.2935	0.0724	200	10.2986
10.2919	0.0905	250	10.2927
10.2835	0.1086	300	10.2894
10.279	0.1267	350	10.2870
10.2818	0.1448	400	10.2860
10.2785	0.1630	450	10.2855
10.2872	0.1811	500	10.2854

Framework versions

• PEFT 0.13.2
• Transformers 4.46.0
• Pytorch 2.5.0+cu124
• Datasets 3.0.1
• Tokenizers 0.20.1