mini-Cogito-R1

Overview

The mini-Cogito-R1 is a lightweight, high-performance language model fine-tuned for text generation, mathematical reasoning, and edge-device optimization. Developed by Daemontatox, this model is based on the unsloth/deepscaler-1.5b-preview architecture and fine-tuned using the Unsloth framework and Huggingface's TRL library, achieving 2x faster training speeds without compromising performance.

---

Key Features

• Efficient Training: Leverages Unsloth for faster and more efficient fine-tuning.
• Optimized for Edge Devices: Designed to run efficiently on resource-constrained devices, making it ideal for edge computing applications.
• Mathematical Reasoning: Excels in tasks requiring logical and mathematical reasoning.
• Text Generation: Capable of generating high-quality, coherent text for a variety of applications.
• Lightweight: Despite its compact size (1.5B parameters), it delivers robust performance.

---

Model Details

• Developed by: Daemontatox
• Model Name: mini-Cogito-R1
• License: Apache-2.0
• Base Model: unsloth/deepscaler-1.5b-preview
• Fine-Tuned From: unsloth/deepscaler-1.5b-preview-unsloth-bnb-4bit
• Framework: Unsloth + Huggingface TRL
• Language: English

---

Training Datasets

The mini-Cogito-R1 model was fine-tuned on a diverse set of high-quality datasets to enhance its reasoning, mathematical, and text-generation capabilities. These datasets include:

1. PrimeIntellect/NuminaMath-QwQ-CoT-5M

   • A large-scale dataset focused on mathematical reasoning and chain-of-thought (CoT) problem-solving.

2. openai/gsm8k

   • A dataset of grade-school math problems designed to test mathematical reasoning and problem-solving skills.

3. cognitivecomputations/dolphin-r1

   • A dataset for instruction-following and reasoning tasks, enhancing the model's ability to understand and execute complex instructions.

4. simplescaling/s1K

   • A lightweight dataset for general-purpose text generation and reasoning tasks.

5. bespokelabs/Bespoke-Stratos-17k

   • A dataset tailored for edge-device optimization and efficient text generation.

---

Use Cases

• Edge Computing: Deploy on edge devices for real-time text generation and reasoning tasks.
• Educational Tools: Assist in solving mathematical problems and logical reasoning exercises.
• Content Creation: Generate high-quality text for blogs, articles, and creative writing.
• Research: Explore efficient training techniques and lightweight model architectures.

---

Performance

The mini-Cogito-R1 was fine-tuned 2x faster using Unsloth's optimized training pipeline, making it a cost-effective solution for developers and researchers. It maintains high accuracy and efficiency, particularly in mathematical reasoning and text generation tasks.

---

How to Use

You can load and use the model with Huggingface's transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Daemontatox/mini-Cogito-R1"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

inputs = tokenizer("Solve for x: 2x + 5 = 15", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))

---

Acknowledgments

• Unsloth Team: For their groundbreaking work on efficient model training.
• Huggingface: For providing the TRL library and ecosystem.
• Open Source Community: For continuous support and contributions.

---

License

This model is licensed under the Apache-2.0 license. For more details, see the LICENSE file.

---

Connect with the Developer

• GitHub: Daemontatox
• Huggingface Model Hub: mini-Cogito-R1

---

---

Dataset References

• NuminaMath-QwQ-CoT-5M: PrimeIntellect
• GSM8K: OpenAI
• Dolphin-R1: Cognitive Computations
• S1K: Simple Scaling
• Bespoke-Stratos-17k: Bespoke Labs

---