bveiseh/phi4-magpie-reasoning-v4-gguf

Phi-4 Magpie Reasoning GGUF v4

This is a GGUF format version of the Phi-4 model fine-tuned on the Magpie dataset (v4).

Model Details

• Base Model: Microsoft Phi-4 (14B parameters)
• Available Formats:
  • GGUF FP16 (full precision)
  • GGUF Q8 (8-bit quantization)
• Fine-tuning: LoRA with merged weights
• Training Dataset: Magpie Reasoning Dataset
• Version: 4

Training Data

• 2,200 excellent quality examples
• 3,000 good quality examples
• Total training samples: 5,200

Evaluation Dataset

• 5 very hard + excellent quality examples
• 5 medium + excellent quality examples
• 5 very easy + excellent quality examples

Technical Details

• LoRA Parameters:

  • Rank (r): 24
  • Alpha: 48
  • Target Modules: q_proj, k_proj, v_proj, o_proj
  • Dropout: 0.05

• Training Configuration:

  • Epochs: 5
  • Learning Rate: 3e-5
  • Batch Size: 1 with gradient accumulation steps of 16
  • Optimizer: AdamW (Fused)
  • Precision: BFloat16 during training
  • Available Formats: FP16 and 8-bit quantized GGUF

Usage with llama.cpp

For CPU inference with the Q8 model:

main -m phi4-magpie-reasoning-q8.gguf -n 512 --repeat_penalty 1.1 --color -i -r User:

For GPU inference with the FP16 model:

main -m phi4-magpie-reasoning-fp16.gguf -n 512 --repeat_penalty 1.1 --color -i -r User: --n-gpu-layers 35

Model Sizes

• GGUF FP16 Format: ~28GB
• GGUF Q8 Format: ~14GB
• Original Model (14B parameters)

License

This model inherits the license terms from Microsoft Phi-4 and the Magpie dataset.