Instructions to use ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF
# Run inference directly in the terminal:
llama cli -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF
# Run inference directly in the terminal:
llama cli -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF
# Run inference directly in the terminal:
./llama-cli -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Use Docker

docker model run hf.co/ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

LM Studio
Jan

vLLM

How to use ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Ollama
How to use ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF with Ollama:
```
ollama run hf.co/ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF
```

Unsloth Studio

How to use ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF to start chatting

How to use ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF with Docker Model Runner:
```
docker model run hf.co/ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF
```

Lemonade

How to use ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Run and chat with the model

lemonade run user.DeepSeek-V4-Flash-0731-REAP25-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Run Hermes

hermes

Atomic Chat

DeepSeek-V4-Flash-0731 — REAP25 2-bit imatrix GGUF (ds4/DwarfStar)

A 25%-expert-pruned, 2-bit imatrix GGUF of deepseek-ai/DeepSeek-V4-Flash-0731, built for the antirez/ds4 (DwarfStar) engine and 96 GB Apple Silicon Macs.


File	`DeepSeek-V4-Flash-0731-REAP25-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf`
Size	68,579,735,164 bytes (63.87 GiB)
Experts	256 → 192 kept in layers 3–42 (REAP25, 25% pruned); layers 0–2 keep 256
Quant	IQ2_XXS (gate/up) + Q2_K (down) routed experts; Q8_0 attention/shared/output; F16 HC/compressor/indexer
Engine	ds4 (`--dspark` optional)
Context	1M (compressed KV: ~8.4 GiB at 1M)

What this is

This is a GGUF-level prune of the stock 2-bit 0731 GGUF: the experts that the REAP25 saliency pruning dropped are removed from the file and the remaining bytes are recompacted. No re-quantization was done — every kept byte is byte-identical to the stock file, so quality is exactly the 2-bit quality of the source, minus the REAP pruning cost. The result is 68.58 GB and fits in 96 GB Macs that cannot hold the 80.8 GiB stock file.

Why the identity of the dropped experts matters (and how we got it)

REAP25 keeps 192 of the 256 experts in layers 3–42. In a GGUF, each expert's weights are stored as a separate contiguous, block-aligned chunk per layer (the expert axis is the slowest tensor dimension, and quant blocks never straddle expert boundaries). Pruning therefore means: for every layer, drop the exact 64 chunks that the REAP saliency run removed — not any 64. If the wrong chunks are dropped, the router addresses experts that are no longer there and the model produces garbage.

The REAP25 MLX weights are compacted to ids 0..191, so the original expert identity is not stored in them. We did not find a published list of which experts were dropped by the REAP saliency run, so we had to work out the mapping ourselves: each REAP25 expert's weights were matched against all 256 stock experts of the same layer on four independent parts (gate/up/down expert weights + router rows). All four parts agreed for 192/192 experts in 40/40 layers, with decisive margins — giving us the exact deletion map used for this file.

Provenance — who did what

Base model — deepseek-ai/DeepSeek-V4-Flash-0731 (304B sparse MoE, 43 layers, 256 routed experts, top-6, 1M ctx).
REAP25 pruning — pipenetwork/DeepSeek-V4-Flash-MLX-REAP25 by PipeNetwork: 25% of routed experts pruned by REAP saliency (layers 0–2 hash-routed keep all 256; layers 3–42 keep 192).
Stock 2-bit GGUF (prune source) — Rednalreden/DeepSeek-V4-Flash-0731-dwarfstar-q2-gguf (86.72 GB): imatrix-guided IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8 quant of the 0731, built with the ds4 toolchain (gguf-tools/deepseek4-quantize).
Expert deletion map (recovered here) — B-exact weight matching, as described above. The map and all scripts are published in the reap-compact-support branch of the ds4 fork (tools/reap25-prune/, docs/REAP25-REPRODUCE.md).
GGUF surgery (this repo) — the 64 pruned experts' quantized chunks were dropped per layer and the file recompacted; reap.* metadata added (reap.enabled, reap.layout=ds4-compact-v1, reap.layer.expert_count, reap.layer.keep_count).
DSpark drafters (optional, third-party) — antirez/deepseek-v4-gguf (DeepSeek-V4-Flash-DSpark-support.gguf) or bleysg/DeepSeek-V4-Flash-DSpark-drafter-GGUF (DSpark-drafter-Q2K-Q8-0731.gguf); load with --mtp <file> --dspark.

Quality & speed (measured on M2 Max 96 GB, ds4-server + llama-benchy)

Perplexity (131,040 wikitext-2-raw-v1 tokens, ds4, same protocol): stock 4.934 → REAP25 5.129, δ +0.195 ppl (+3.95%) — better than pipenetwork's published +4.5% at 4-bit (different protocol). A model property — engine updates do not change it.
Speed (pp 2048, tg 128, fixed 1M-ctx server; engine = the reap-compact-support branch at 01bdc35, i.e. upstream DS4 rebased onto 2026-08-09 main 84cc882 with Ivan Fioravanti's exact pre-M5 Metal decode fusions, 4e401a1 et al.):

Context depth Prefill (PP) t/s Decode (TG) t/s

1K 183.6 31.7

8K 192.0 35.2

32K 111.0 15.6*

128K 93.2 24.5

*Samples are single-shot unless noted; expect ±a few %. The 32K decode sample was visibly stalled (mean 15.6 t/s vs peak 25.0); two same-session standard-build samples with the MTP head attached measured 29.96 / 30.46 t/s at 32K, so the true 32K decode is ≈28–30 t/s. 8K reproduced identically (35.19) across two separate server sessions.

The upstream long-context decode fix + pre-M5 fusions roughly doubled deep-context decode versus the pre-rebase engine (128K TG 11.23 → 24.47, +118%); 1K decode is unchanged (~32 t/s) and prefill is somewhat lower on this quantization path. Earlier Aug 3/4 numbers (TG 14.7/11.7/11.2/11.5/11.7 at 1K–128K) are superseded.

Context depth	Prefill (PP) t/s	Decode (TG) t/s
1K	183.6	31.7
8K	192.0	35.2
32K	111.0	15.6*
128K	93.2	24.5

Reproduction

Full recipe (map recovery + GGUF surgery + validation): docs/REAP25-REPRODUCE.md in the reap-compact-support branch — DESIGN notes + RUNBOOK + scripts (bexact.py, prune_gguf.py, extract_tables.py, keep_map_bexact.json).

Notes

ds4's GGUF variant differs from llama.cpp's (u64 string lengths, no size field, 32-byte data alignment) — this file is for the ds4 engine, not llama.cpp.
Not an official DeepSeek release; pruning + 2-bit reduce quality. Evaluate before production use.
DSpark speculative decode (--mtp <drafter> --dspark) is linked for completeness but measures as a net slowdown on the M2 Max: the drafters predict the pruned model well (~69% acceptance), yet proposal+verify overhead exceeds the decode they save at every depth. Run without --dspark on this class of hardware.

Credits and acknowledgements

This is small work sitting on top of a very large body of work by many people, and we want to thank them properly:

DeepSeek — for the DeepSeek-V4-Flash-0731 base model, the DSpark draft stages, and the reference inference implementation.
antirez (Salvatore Sanfilippo) — for the ds4 (DwarfStar) engine, its gguf-tools/deepseek4-quantize toolchain and imatrix, the REAP-compact runtime support this file is built for, and the DeepSeek-V4-Flash-DSpark-support.gguf drafter.
PipeNetwork — for the DeepSeek-V4-Flash-MLX-REAP25 REAP pruning and the from-scratch MLX port of the architecture.
Rednalreden — for the stock 2-bit imatrix GGUF we pruned from.
GaelicThunder — reference 2-bit builds of the 0731 that guided the quant recipe.
eouya2 — prior compact REAP25 GGUF work that proved the GGUF-level expert-subset approach works.
0xSero — REAP observation datasets and the 0731 REAP reference checkpoints we used to sanity-check our map recovery.
bleysg — for the DSpark-drafter-Q2K-Q8-0731.gguf.
ggerganov / llama.cpp — for the IQ2_XXS/Q2_K dequant reference tables used in the map recovery.

If you are one of the people above and we misrepresented your work, please open an issue or pull request on the reap-compact-support branch and we will fix it promptly.

Downloads last month: 82

GGUF

Model size

220B params

Architecture

deepseek4

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF

Base model

deepseek-ai/DeepSeek-V4-Flash-0731

Quantized

(125)

this model