Triangle104/Open-Reasoner-Zero-7B-Q5_K_M-GGUF

This model was converted to GGUF format from Open-Reasoner-Zero/Open-Reasoner-Zero-7B using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

---

An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model

Overview

🌊 We introduce Open-Reasoner-Zero, the first open source implementation of large-scale reasoning-oriented RL training focusing on scalability, simplicity and accessibility.

To enable broader participation in this pivotal moment we witnessed and accelerate research towards artificial general intelligence (AGI), we release our source code, parameter settings, training data, and model weights. Please refer to our paper for more insights.

Let the Reasoner-Zero tide rise!

    Releases 📦

[2025/02/18] We release Open-Reasoner-Zero.

As part of this release, we open-source:

🌊 Paper on our comprehensive analysis and insights in Reasoner-Zero training 🤗 HF Model Open-Reasoner-Zero-7B and Open-Reasoner-Zero-32B 🎁 Our curated 57k training data 📄 Training Scripts to enjoy your own Reasoner-Zero journey!

    Key Features in Codebase 🔑

Adopt single controller trainer design, flexible and researcher-friendly. Colocate training and generation in the same GPUs to maximize GPU utilization.

    Getting Started 🚀






    


    Installation & Training Scripts

We release our Dockerfile in docker folder to facilitate the reproducibility of our training.

To install the package, run:

pip install -e .

    Start Orz-7B PPO Training

debug running command in single node:

DEBUG_MODE=True python -m playground.orz_7b_ppo

Multi-node Training:

first on master node, run:

ray start --head

then on other nodes, run:

ray start --address=':'

then on master node, run:

python -m playground.orz_7b_ppo

Your training log will be shown in the master node terminal.

    Start Orz-32B PPO Training

running command in 8 nodes:

first on master node, run:

ray start --head

then on other nodes, run:

ray start --address=':'

then on master node, run:

python -m playground.orz_32b_ppo

Your training log will be shown in the master node terminal.

    Data

We release all of 57k curated high-quality training data in the data folder.

The details for how to collect data are described in our paper.

    Acknowledgements

This work was supported by computing resources and valuable feedback provided by StepFun and Tsinghua University. Our training framework is built on OpenRLHF, vllm, DeepSpeed and ray. Our model is based on Qwen2.5-7B and Qwen2.5-32B. We thank Project Numina and Tulu3 for their collected open sourced data.

---

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo Triangle104/Open-Reasoner-Zero-7B-Q5_K_M-GGUF --hf-file open-reasoner-zero-7b-q5_k_m.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo Triangle104/Open-Reasoner-Zero-7B-Q5_K_M-GGUF --hf-file open-reasoner-zero-7b-q5_k_m.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo Triangle104/Open-Reasoner-Zero-7B-Q5_K_M-GGUF --hf-file open-reasoner-zero-7b-q5_k_m.gguf -p "The meaning to life and the universe is"

or

./llama-server --hf-repo Triangle104/Open-Reasoner-Zero-7B-Q5_K_M-GGUF --hf-file open-reasoner-zero-7b-q5_k_m.gguf -c 2048