Instructions to use apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 # Run inference directly in the terminal: llama cli -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 # Run inference directly in the terminal: llama cli -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 # Run inference directly in the terminal: ./llama-cli -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 # Run inference directly in the terminal: ./build/bin/llama-cli -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
Use Docker
docker model run hf.co/apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
- LM Studio
- Jan
- vLLM
How to use apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
- Ollama
How to use apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 with Ollama:
ollama run hf.co/apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
- Unsloth Studio
How to use apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 to start chatting
- Pi
How to use apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128" } ] } } }Run Pi
# Start Pi in your project directory: pi
- OpenClaw new
How to use apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 with Docker Model Runner:
docker model run hf.co/apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
- Lemonade
How to use apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
Run and chat with the model
lemonade run user.DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128
Run Hermes
hermes
- Atomic Chat
| { | |
| "schema": "deepseek-model-tools.build128.v1", | |
| "artifact": "ds4-quality128", | |
| "output": "DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128", | |
| "engine": "ds4", | |
| "dspark": true, | |
| "created_at": "2026-08-01T13:29:21.384605+00:00", | |
| "source": { | |
| "path": "apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8", | |
| "only_weight_source": true, | |
| "config_sha256": "6c8f3d2d3b48707541b88f32f22ef3f0f8a6b57d8523281e2b8d3cdb0ae9a023", | |
| "index_sha256": "98efab455cf08dfbbbaaba6f570e1bf10bf927d2b4c3c453a59c2f6f0e3be92b", | |
| "abliteration_manifest_sha256": "8e6f40f3d92542720aabdafb728f90e3397fba68c6a8d7ba91772b5dd41bc767", | |
| "edited_tensors": 36, | |
| "upstream_model": "deepseek-ai/DeepSeek-V4-Flash-0731", | |
| "upstream_revision": "9e165c30e2704aec5d9d593cce3eebd58bbef1cb", | |
| "weight_shards": [ | |
| { | |
| "filename": "model-00001-of-00048.safetensors", | |
| "bytes": 1059061856, | |
| "sha256": "f3668ba4cccf1ca6a7eb84e888fb92c1cdc7204d472ba9db771e6fd3abf6b874", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00002-of-00048.safetensors", | |
| "bytes": 3566321192, | |
| "sha256": "77b26c939a0e25b3113c8d6bb04e1901a748bd4a7d2589e3bfdaabdf1e9bba14", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00003-of-00048.safetensors", | |
| "bytes": 3566321192, | |
| "sha256": "412abf4c906faadc221ef0cb50f90fe20bde8454a08ad4dc2364b6b79e7fda5c", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00004-of-00048.safetensors", | |
| "bytes": 3596229272, | |
| "sha256": "9610f56bc587fb0ff9a8b68a60299482ee8c433fe5b5587e4257aca98add4a2e", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00005-of-00048.safetensors", | |
| "bytes": 3568768976, | |
| "sha256": "f87a5ac7b8becc31f9c3169afd3a6f33fb82b4af9e21022e3755a10bc28f0180", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00006-of-00048.safetensors", | |
| "bytes": 3590024776, | |
| "sha256": "4a4f3764e3fc772b9fba67f0a44ef68e18f178b6f00faa80b75db549e51894cd", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00007-of-00048.safetensors", | |
| "bytes": 3568768976, | |
| "sha256": "df81bb80e27a689e01fa579eebd6499f86e0b6105f7fea18961aa5eebbbee9bc", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00008-of-00048.safetensors", | |
| "bytes": 3590024776, | |
| "sha256": "224968d2b27f8669365ec08657a768dfec40da0585f85f302a31495931f6a526", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00009-of-00048.safetensors", | |
| "bytes": 3568768976, | |
| "sha256": "04d69ef1071fff8721c62968c200a5583122b59b015e9ef9b2978bfed271b2b7", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00010-of-00048.safetensors", | |
| "bytes": 3590024776, | |
| "sha256": "627145f4ebeb1cc3f5bdd03416b8cb7370b3c96974853cbeb8e5516ad5713e49", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00011-of-00048.safetensors", | |
| "bytes": 3568768976, | |
| "sha256": "e4b8e601dcbebe902e0102e7b098b670a121cb8b9564dd719fc41d782c8416e0", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00012-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "d200bcd9186f15344b8271f845bd81f782509c5997ee493b1a96079e91a1a8ac", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00013-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "9c36e454195177bafdb866bbfd052f69dd727fafbe328c0d5d4d659ab49389b2", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00014-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "ce44ce426be0ed8b21141ac1659ca108d2d0705b5b442a7691511bc38831bf9e", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00015-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "f9b40ffad55e01cbfc012c737640f882ad7161026ea1c6d878d272c655e74b02", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00016-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "acf5cbaf18a2d2e005358b0a81527ed70bc92f352ab46f658172a0505983bb2c", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00017-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "d67716007b48845ed1de1473ca0ab15908a9dff8b592bb62ab95137cc67cf198", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00018-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "ac550ef17db3b8c95c2a7858fefa7ba08c94b5e6feafa8bfdb3f9763612fd3b1", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00019-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "c31a222ce5c7e0d02b8c2614badd16781cb3979f7e516d5293ab976ee670f34f", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00020-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "a27655e314a1919d9c7042b1c7f842fa8c8f1bf77df4c0a9586371b4ab3029d9", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00021-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "4068e1c0a59dcd4b7a22c09cf870e86439407f1eb8c8669b20d81d3294e4ca60", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00022-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "eb87aef24cbf17c98f8ab4803590606db89acc69c83b9d4f44919bac328d830d", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00023-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "4b8d427c9d2f285986d18b465f3ffcb73c12a109e1dedc8b0a3b6ac83e0e1ab7", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00024-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "2cf277a128b4f51dcd13c130db18e325006962eec0a1f1e90596d32085980daa", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00025-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "dd813e4837baa77bd5fac84d5dce225e6e949c7e781cf40621c06081b6bfcbe1", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00026-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "a4c8f53dbf43374c5a277e992973b0f5b9e916022dd17cecc024c4290165eae8", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00027-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "70050f03122a0050bad34b03eb0bb88def42f2c3ea99c525c1af776409efb76e", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00028-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "51113c3fce251a429452829adea388497184a80e0c0023febbb88c4068f65842", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00029-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "f7b9198bb1e6c05ab5e88bf22fbe78d3f67f789a494d4c1be763787ee11f5f1e", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00030-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "51e9984913ff3abbb7649d8738e764360c165401a9c31d9b53715c1fbe313061", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00031-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "d31d4f707391eca341eaa6025f3e8ecf21ee254340bd8f11d494200aa8923447", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00032-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "b7ecad0122f051d1e2a0953312640ed53ac7da3875766e1e70ff0d0666c28087", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00033-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "7bf5a6c9a2774698ebf819f3abf20604376fd75daecb6ecb6499df964e4447af", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00034-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "b0de49f523078e8766c5deb61be4fc73da35271f3bd354bb3e7c71e975cd8be5", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00035-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "471a793bb28aa16c95d0cad6e9194191fd9c2e0657c9e35917bd7856ddbf8187", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00036-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "002bdca78f533708438ed04ca42139fb67f0325571adb29645f5ef38be54b066", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00037-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "71f148a710c0ff1b505d6e285b1e782eef87d14d5fa4f5411e1faa4c06210708", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00038-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "ab90e440639d4b0de9a9e2f7a48238e7ccdc08c37d40cf91d1a380ba26c1938a", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00039-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "26f3acc9d23d426edabef80260ac4030aad030553356c47b1d5d47667c0813c3", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00040-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "549d59fc74aa7bba55678da85e5a369ebe8017763c4d5f83359e15323659afcf", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00041-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "1868317b4ff748d1388805c530fc7b0310397fdef8ac1c0bd7f034f14abe83ce", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00042-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "11e81dc83b2f6ab688b9205552a147cf781d73ee791a8d54c763e0a461006def", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00043-of-00048.safetensors", | |
| "bytes": 3568770544, | |
| "sha256": "bc1dc626c7ec234f58807443ecbdec2a5927be771339697cbac3dca94c6ed9a0", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00044-of-00048.safetensors", | |
| "bytes": 3590026352, | |
| "sha256": "a20dfe1e58e2744ed8c82cdca63326ed62c2f024361a072ecb827edd9e0287bc", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00045-of-00048.safetensors", | |
| "bytes": 1059332516, | |
| "sha256": "a5be6aed7b84fc87ec42b5d24ba0b0d67f253a3906fcd99c13f4f7be5958fc00", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00046-of-00048.safetensors", | |
| "bytes": 3610455184, | |
| "sha256": "da1e0108b463653b292a648aef58b4dbbeac03a8c213c53829b99c6d84d941c9", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00047-of-00048.safetensors", | |
| "bytes": 3560111960, | |
| "sha256": "b8b41236c1ccf4378a3ae2ccc22a784ca1a5681f13a1d64c262836b788a0f093", | |
| "upload_mode": "lfs" | |
| }, | |
| { | |
| "filename": "model-00048-of-00048.safetensors", | |
| "bytes": 3692775244, | |
| "sha256": "d8b68325b29a33f7ee43cafcaa69c5de8c1de7c1f6007758961ad4ed11aa74db", | |
| "upload_mode": "lfs" | |
| } | |
| ], | |
| "weight_shard_bytes": 166886535336, | |
| "weight_shard_hashes_verified": true, | |
| "weight_shard_hashes_verified_at": "2026-08-01T11:53:46.973992+00:00", | |
| "post_hash_stat_snapshot": { | |
| "model-00001-of-00048.safetensors": { | |
| "bytes": 1059061856, | |
| "mtime_ns": 1785506342100000000, | |
| "ctime_ns": 1785506342100000000, | |
| "device": 16777242, | |
| "inode": 10534358 | |
| }, | |
| "model-00002-of-00048.safetensors": { | |
| "bytes": 3566321192, | |
| "mtime_ns": 1785506344340000000, | |
| "ctime_ns": 1785506344340000000, | |
| "device": 16777242, | |
| "inode": 10538402 | |
| }, | |
| "model-00003-of-00048.safetensors": { | |
| "bytes": 3566321192, | |
| "mtime_ns": 1785506346560000000, | |
| "ctime_ns": 1785506346560000000, | |
| "device": 16777242, | |
| "inode": 10552010 | |
| }, | |
| "model-00004-of-00048.safetensors": { | |
| "bytes": 3596229272, | |
| "mtime_ns": 1785506349630000000, | |
| "ctime_ns": 1785506349630000000, | |
| "device": 16777242, | |
| "inode": 10565618 | |
| }, | |
| "model-00005-of-00048.safetensors": { | |
| "bytes": 3568768976, | |
| "mtime_ns": 1785506351960000000, | |
| "ctime_ns": 1785506351960000000, | |
| "device": 16777242, | |
| "inode": 10579340 | |
| }, | |
| "model-00006-of-00048.safetensors": { | |
| "bytes": 3590024776, | |
| "mtime_ns": 1785506354260000000, | |
| "ctime_ns": 1785506354260000000, | |
| "device": 16777242, | |
| "inode": 10592957 | |
| }, | |
| "model-00007-of-00048.safetensors": { | |
| "bytes": 3568768976, | |
| "mtime_ns": 1785506356530000000, | |
| "ctime_ns": 1785506356530000000, | |
| "device": 16777242, | |
| "inode": 10606655 | |
| }, | |
| "model-00008-of-00048.safetensors": { | |
| "bytes": 3590024776, | |
| "mtime_ns": 1785506358750000000, | |
| "ctime_ns": 1785506358750000000, | |
| "device": 16777242, | |
| "inode": 10620272 | |
| }, | |
| "model-00009-of-00048.safetensors": { | |
| "bytes": 3568768976, | |
| "mtime_ns": 1785506362250000000, | |
| "ctime_ns": 1785506362250000000, | |
| "device": 16777242, | |
| "inode": 10633970 | |
| }, | |
| "model-00010-of-00048.safetensors": { | |
| "bytes": 3590024776, | |
| "mtime_ns": 1785506365890000000, | |
| "ctime_ns": 1785506365890000000, | |
| "device": 16777242, | |
| "inode": 10647587 | |
| }, | |
| "model-00011-of-00048.safetensors": { | |
| "bytes": 3568768976, | |
| "mtime_ns": 1785506369620000000, | |
| "ctime_ns": 1785506369620000000, | |
| "device": 16777242, | |
| "inode": 10661285 | |
| }, | |
| "model-00012-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506377140000000, | |
| "ctime_ns": 1785506377140000000, | |
| "device": 16777242, | |
| "inode": 10674902 | |
| }, | |
| "model-00013-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506386820000000, | |
| "ctime_ns": 1785506386820000000, | |
| "device": 16777242, | |
| "inode": 10688602 | |
| }, | |
| "model-00014-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506396120000000, | |
| "ctime_ns": 1785506396120000000, | |
| "device": 16777242, | |
| "inode": 10702221 | |
| }, | |
| "model-00015-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506405630000000, | |
| "ctime_ns": 1785506405630000000, | |
| "device": 16777242, | |
| "inode": 10715921 | |
| }, | |
| "model-00016-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506415210000000, | |
| "ctime_ns": 1785506415210000000, | |
| "device": 16777242, | |
| "inode": 10729540 | |
| }, | |
| "model-00017-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506424840000000, | |
| "ctime_ns": 1785506424840000000, | |
| "device": 16777242, | |
| "inode": 10743240 | |
| }, | |
| "model-00018-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506432280000000, | |
| "ctime_ns": 1785506432280000000, | |
| "device": 16777242, | |
| "inode": 10756859 | |
| }, | |
| "model-00019-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506439960000000, | |
| "ctime_ns": 1785506439960000000, | |
| "device": 16777242, | |
| "inode": 10770559 | |
| }, | |
| "model-00020-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506449550000000, | |
| "ctime_ns": 1785506449550000000, | |
| "device": 16777242, | |
| "inode": 10784178 | |
| }, | |
| "model-00021-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506459270000000, | |
| "ctime_ns": 1785506459270000000, | |
| "device": 16777242, | |
| "inode": 10797878 | |
| }, | |
| "model-00022-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506468940000000, | |
| "ctime_ns": 1785506468940000000, | |
| "device": 16777242, | |
| "inode": 10811497 | |
| }, | |
| "model-00023-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506478610000000, | |
| "ctime_ns": 1785506478610000000, | |
| "device": 16777242, | |
| "inode": 10825197 | |
| }, | |
| "model-00024-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506488130000000, | |
| "ctime_ns": 1785506488130000000, | |
| "device": 16777242, | |
| "inode": 10838816 | |
| }, | |
| "model-00025-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506495770000000, | |
| "ctime_ns": 1785506495770000000, | |
| "device": 16777242, | |
| "inode": 10852516 | |
| }, | |
| "model-00026-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506505150000000, | |
| "ctime_ns": 1785506505150000000, | |
| "device": 16777242, | |
| "inode": 10866135 | |
| }, | |
| "model-00027-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506514650000000, | |
| "ctime_ns": 1785506514650000000, | |
| "device": 16777242, | |
| "inode": 10879835 | |
| }, | |
| "model-00028-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506524000000000, | |
| "ctime_ns": 1785506524000000000, | |
| "device": 16777242, | |
| "inode": 10893454 | |
| }, | |
| "model-00029-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506533430000000, | |
| "ctime_ns": 1785506533430000000, | |
| "device": 16777242, | |
| "inode": 10907154 | |
| }, | |
| "model-00030-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506540820000000, | |
| "ctime_ns": 1785506540820000000, | |
| "device": 16777242, | |
| "inode": 10920773 | |
| }, | |
| "model-00031-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506550210000000, | |
| "ctime_ns": 1785506550210000000, | |
| "device": 16777242, | |
| "inode": 10934473 | |
| }, | |
| "model-00032-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506559630000000, | |
| "ctime_ns": 1785506559630000000, | |
| "device": 16777242, | |
| "inode": 10948092 | |
| }, | |
| "model-00033-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506567220000000, | |
| "ctime_ns": 1785506567220000000, | |
| "device": 16777242, | |
| "inode": 10961792 | |
| }, | |
| "model-00034-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506576510000000, | |
| "ctime_ns": 1785506576510000000, | |
| "device": 16777242, | |
| "inode": 10975411 | |
| }, | |
| "model-00035-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506585920000000, | |
| "ctime_ns": 1785506585920000000, | |
| "device": 16777242, | |
| "inode": 10989111 | |
| }, | |
| "model-00036-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506595350000000, | |
| "ctime_ns": 1785506595350000000, | |
| "device": 16777242, | |
| "inode": 11002730 | |
| }, | |
| "model-00037-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506602780000000, | |
| "ctime_ns": 1785506602780000000, | |
| "device": 16777242, | |
| "inode": 11016430 | |
| }, | |
| "model-00038-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506611980000000, | |
| "ctime_ns": 1785506611980000000, | |
| "device": 16777242, | |
| "inode": 11030049 | |
| }, | |
| "model-00039-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506621720000000, | |
| "ctime_ns": 1785506621720000000, | |
| "device": 16777242, | |
| "inode": 11043749 | |
| }, | |
| "model-00040-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506630860000000, | |
| "ctime_ns": 1785506630860000000, | |
| "device": 16777242, | |
| "inode": 11057368 | |
| }, | |
| "model-00041-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506638120000000, | |
| "ctime_ns": 1785506638120000000, | |
| "device": 16777242, | |
| "inode": 11071068 | |
| }, | |
| "model-00042-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506647820000000, | |
| "ctime_ns": 1785506647820000000, | |
| "device": 16777242, | |
| "inode": 11084687 | |
| }, | |
| "model-00043-of-00048.safetensors": { | |
| "bytes": 3568770544, | |
| "mtime_ns": 1785506657140000000, | |
| "ctime_ns": 1785506657140000000, | |
| "device": 16777242, | |
| "inode": 11098387 | |
| }, | |
| "model-00044-of-00048.safetensors": { | |
| "bytes": 3590026352, | |
| "mtime_ns": 1785506666700000000, | |
| "ctime_ns": 1785506666700000000, | |
| "device": 16777242, | |
| "inode": 11112006 | |
| }, | |
| "model-00045-of-00048.safetensors": { | |
| "bytes": 1059332516, | |
| "mtime_ns": 1785506668200000000, | |
| "ctime_ns": 1785506668200000000, | |
| "device": 16777242, | |
| "inode": 11125706 | |
| }, | |
| "model-00046-of-00048.safetensors": { | |
| "bytes": 3610455184, | |
| "mtime_ns": 1785506675660000000, | |
| "ctime_ns": 1785506675660000000, | |
| "device": 16777242, | |
| "inode": 11129751 | |
| }, | |
| "model-00047-of-00048.safetensors": { | |
| "bytes": 3560111960, | |
| "mtime_ns": 1785506683250000000, | |
| "ctime_ns": 1785506683250000000, | |
| "device": 16777242, | |
| "inode": 11143529 | |
| }, | |
| "model-00048-of-00048.safetensors": { | |
| "bytes": 3692775244, | |
| "mtime_ns": 1785506690860000000, | |
| "ctime_ns": 1785506690860000000, | |
| "device": 16777242, | |
| "inode": 11157115 | |
| } | |
| } | |
| }, | |
| "profile": { | |
| "path": "embedded-profile", | |
| "source_profile_sha256": "6c25185500c07d4419516d30b0902e457c99bd70543c4f5bea026756d1804eb4", | |
| "content": { | |
| "name": "dsv4-ds4-quality128-native-mxfp4", | |
| "engine": "ds4", | |
| "description": "Quality-first 128 GB DS4 profile preserving exact native MXFP4 routed experts on ten sensitivity-selected layers, with IQ2_XXS/Q2_K elsewhere and native-MXFP4 DSpark down projections.", | |
| "source": "apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8", | |
| "output": "DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128", | |
| "ds4_revision": "45e6cd76e849d86a8280e24ef43fa149ce9e330a", | |
| "toolchain": { | |
| "root": "antirez/ds4@ds4f-mxfp4", | |
| "upstream_base_revision": "4893e0c40fba03dbc85555faeb035799aa04e0b6", | |
| "quantizer_sha256": "f0a381f4ada808ea2afa740d964354fa327fc1235ba7cebf50874eb89fb97ac5", | |
| "runtime_sha256": "2aaf20469b9918d6d6ab8787a02811c11228547cd979787879a03dba8a9e7824" | |
| }, | |
| "main": { | |
| "experts": "iq2_xxs", | |
| "routed_w2": "q2_k", | |
| "attention_proj": "q8_0", | |
| "shared": "q8_0", | |
| "preserve_indexer_f16": true, | |
| "tensor_types": [ | |
| "output.weight=q8_0" | |
| ], | |
| "native_mxfp4_layers": [ | |
| 10, | |
| 14, | |
| 30, | |
| 34, | |
| 37, | |
| 38, | |
| 39, | |
| 40, | |
| 41, | |
| 42 | |
| ], | |
| "native_mxfp4_layer_selection": "Antirez 37-42 seed plus cross-recipe sensitivity controls 10,14,30,34; source packed I8 codes and F8_E8M0 scales are preserved exactly.", | |
| "estimated_bytes": 102826238912 | |
| }, | |
| "dspark": { | |
| "experts": "iq2_xxs", | |
| "routed_w2": "mxfp4", | |
| "block_size": 5, | |
| "target_layers": [ | |
| 40, | |
| 41, | |
| 42 | |
| ], | |
| "markov_rank": 256, | |
| "noise_token_id": 128799, | |
| "calibration": "target-layer proxy aliases" | |
| }, | |
| "verification": { | |
| "require_imatrix_strict": true, | |
| "require_native_byte_compare": true, | |
| "main": { | |
| "bytes": 102826238912, | |
| "tensors": 1328, | |
| "tensor_types": { | |
| "i32": 3, | |
| "iq2_xxs": 66, | |
| "q2_k": 33, | |
| "mxfp4": 30, | |
| "f16": 359, | |
| "f32": 492, | |
| "q8_0": 345 | |
| } | |
| }, | |
| "dspark": { | |
| "bytes": 7297737120, | |
| "tensors": 81, | |
| "tensor_types": { | |
| "f32": 34, | |
| "f16": 7, | |
| "q8_0": 31, | |
| "iq2_xxs": 6, | |
| "mxfp4": 3 | |
| } | |
| } | |
| }, | |
| "calibration": { | |
| "repo_id": "ox-ox/DeepSeek-V4-Flash-0731-GGUF", | |
| "revision": "6d58a3a36030c3ccb969bb5759fc6ae08cd299f8", | |
| "filename": "imatrix/DeepSeek-V4-Flash-0731-chat-v2-routed-moe-ds4-1p5m.dat", | |
| "bytes": 450892654, | |
| "sha256": "6fce7674df701de544e5d3351aab04e67602eddeafeb48cf70e77ebe47239eb4", | |
| "coverage_policy": "exact 129 routed names and vector dimensions; strict mode checks only imatrix-consuming output types" | |
| }, | |
| "template": { | |
| "role": "metadata-tokenizer-tensor-order-only", | |
| "repo_id": "antirez/deepseek-v4-gguf", | |
| "revision": "1cd7b564460821938add0475a60b942c409295e0", | |
| "filename": "DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf", | |
| "remote_bytes": 86720111488, | |
| "lfs_sha256": "ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0", | |
| "xet_hash": "7da16e1025c1b856490c29c341f4e467d15cb195389c70383dade5e6108799ac", | |
| "header_sha256": "f0e1d5e8f3b008402aa6eb32cada3873dd926c8bf5e7d00d7788eec65f09dd6d" | |
| }, | |
| "recipe_control": { | |
| "repo_id": "antirez/deepseek-v4-gguf", | |
| "revision": "1cd7b564460821938add0475a60b942c409295e0", | |
| "dspark_filename": "DeepSeek-V4-Flash-DSpark-support.gguf", | |
| "weights_used": false | |
| } | |
| } | |
| }, | |
| "calibration": { | |
| "repo_id": "ox-ox/DeepSeek-V4-Flash-0731-GGUF", | |
| "filename": "imatrix/DeepSeek-V4-Flash-0731-chat-v2-routed-moe-ds4-1p5m.dat", | |
| "revision": "6d58a3a36030c3ccb969bb5759fc6ae08cd299f8", | |
| "bytes": 450892654, | |
| "sha256": "6fce7674df701de544e5d3351aab04e67602eddeafeb48cf70e77ebe47239eb4", | |
| "coverage_validation": { | |
| "entries": 129, | |
| "scope": "all routed gate/up/down tensors", | |
| "policy": "exact-name-and-vector-length", | |
| "global_ds4_imatrix_strict": true | |
| } | |
| }, | |
| "template": { | |
| "repo_id": "antirez/deepseek-v4-gguf", | |
| "filename": "DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf", | |
| "revision": "1cd7b564460821938add0475a60b942c409295e0", | |
| "role": "metadata-tokenizer-tensor-order-and-shapes-only", | |
| "weights_used": false, | |
| "remote_bytes": 86720111488, | |
| "lfs_sha256": "ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0", | |
| "xet_hash": "7da16e1025c1b856490c29c341f4e467d15cb195389c70383dade5e6108799ac", | |
| "downloaded_header_bytes": 67108864, | |
| "header_sha256": "f0e1d5e8f3b008402aa6eb32cada3873dd926c8bf5e7d00d7788eec65f09dd6d", | |
| "resolved": { | |
| "repo_id": "antirez/deepseek-v4-gguf", | |
| "filename": "DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf", | |
| "role": "metadata-tokenizer-tensor-order-and-shapes-only", | |
| "weights_used": false, | |
| "remote_bytes": 86720111488, | |
| "lfs_sha256": "ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0", | |
| "xet_hash": "7da16e1025c1b856490c29c341f4e467d15cb195389c70383dade5e6108799ac", | |
| "resolved_etag": "7da16e1025c1b856490c29c341f4e467d15cb195389c70383dade5e6108799ac", | |
| "range": "bytes=0-67108863", | |
| "header_bytes": 67108864, | |
| "header_sha256": "f0e1d5e8f3b008402aa6eb32cada3873dd926c8bf5e7d00d7788eec65f09dd6d" | |
| } | |
| }, | |
| "dspark_calibration_alias": { | |
| "schema": "deepseek-model-tools.dspark-imatrix-alias.v1", | |
| "function": "extend_dspark_legacy_imatrix", | |
| "source": { | |
| "repo_id": "ox-ox/DeepSeek-V4-Flash-0731-GGUF", | |
| "revision": "6d58a3a36030c3ccb969bb5759fc6ae08cd299f8", | |
| "filename": "imatrix/DeepSeek-V4-Flash-0731-chat-v2-routed-moe-ds4-1p5m.dat", | |
| "bytes": 450892654, | |
| "sha256": "6fce7674df701de544e5d3351aab04e67602eddeafeb48cf70e77ebe47239eb4" | |
| }, | |
| "output_sha256": "689b446ed2e2657ebcb69a6516781ee0444e07641aeb6125233fb4e6fe7cbce3", | |
| "mapping": [ | |
| { | |
| "stage": 0, | |
| "target_layer": 40, | |
| "alias": "mtp.0.ffn_gate_exps.weight", | |
| "source": "blk.40.ffn_gate_exps.weight" | |
| }, | |
| { | |
| "stage": 0, | |
| "target_layer": 40, | |
| "alias": "mtp.0.ffn_up_exps.weight", | |
| "source": "blk.40.ffn_up_exps.weight" | |
| }, | |
| { | |
| "stage": 0, | |
| "target_layer": 40, | |
| "alias": "mtp.0.ffn_down_exps.weight", | |
| "source": "blk.40.ffn_down_exps.weight" | |
| }, | |
| { | |
| "stage": 1, | |
| "target_layer": 41, | |
| "alias": "mtp.1.ffn_gate_exps.weight", | |
| "source": "blk.41.ffn_gate_exps.weight" | |
| }, | |
| { | |
| "stage": 1, | |
| "target_layer": 41, | |
| "alias": "mtp.1.ffn_up_exps.weight", | |
| "source": "blk.41.ffn_up_exps.weight" | |
| }, | |
| { | |
| "stage": 1, | |
| "target_layer": 41, | |
| "alias": "mtp.1.ffn_down_exps.weight", | |
| "source": "blk.41.ffn_down_exps.weight" | |
| }, | |
| { | |
| "stage": 2, | |
| "target_layer": 42, | |
| "alias": "mtp.2.ffn_gate_exps.weight", | |
| "source": "blk.42.ffn_gate_exps.weight" | |
| }, | |
| { | |
| "stage": 2, | |
| "target_layer": 42, | |
| "alias": "mtp.2.ffn_up_exps.weight", | |
| "source": "blk.42.ffn_up_exps.weight" | |
| }, | |
| { | |
| "stage": 2, | |
| "target_layer": 42, | |
| "alias": "mtp.2.ffn_down_exps.weight", | |
| "source": "blk.42.ffn_down_exps.weight" | |
| } | |
| ], | |
| "tool": { | |
| "repository_revision": "d97ba6b63ed857a5456f92a5190381fd7e45ca9f", | |
| "importer_sha256": "b39987ce5ce35564bb4053ec11265c0ff41be233595ae17a6e039318a52dd4da" | |
| }, | |
| "coverage_validation": { | |
| "entries": 138, | |
| "trunk_entries": 129, | |
| "dspark_entries": 9, | |
| "coverage": "exact-name-and-vector-length" | |
| } | |
| }, | |
| "tools": { | |
| "ds4_revision": "d516d4eeb82c454aeb2831af1b1961801d6b571b", | |
| "ds4_root": "antirez/ds4@ds4f-mxfp4", | |
| "ds4_quantizer_sha256": "f0a381f4ada808ea2afa740d964354fa327fc1235ba7cebf50874eb89fb97ac5", | |
| "ds4_runtime_sha256": "2aaf20469b9918d6d6ab8787a02811c11228547cd979787879a03dba8a9e7824", | |
| "omlx_revision": "76352ed2363e42b2146243463875756e683a640d", | |
| "deepseek_model_tools_revision": "e24746463a0a3e79036dd9c0472deac6ce704f08", | |
| "deepseek_model_tools_worktree": { | |
| "dirty": true, | |
| "status_sha256": "59efc8a4b51d3dc33f267055db94edc9ac8e2392aa9f6e7033b3842a060b4ad2", | |
| "tracked_patch_sha256": "658a58301aad033c5027f62ee1f1719ab0b01141c4b06b2d296f5f05c8f76aea" | |
| }, | |
| "build_driver_sha256": "e9048f7535fe360756ce4505edae8f5d10f901e3fcc7759a37abbfbfc3949b4e" | |
| }, | |
| "commands": [ | |
| [ | |
| "deepseek4-quantize", | |
| "--hf", | |
| "apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8", | |
| "--template", | |
| "antirez/deepseek-v4-gguf/ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0.header-only.sparse.gguf", | |
| "--experts", | |
| "iq2_xxs", | |
| "--routed-w2", | |
| "q2_k", | |
| "--attention-proj", | |
| "q8_0", | |
| "--shared", | |
| "q8_0", | |
| "--threads", | |
| "8", | |
| "--imatrix", | |
| "ox-ox/DeepSeek-V4-Flash-0731-GGUF/imatrix/DeepSeek-V4-Flash-0731-chat-v2-routed-moe-ds4-1p5m.dat", | |
| "--tensor-type", | |
| "output.weight=q8_0", | |
| "--tensor-type", | |
| "blk.2.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.4.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.6.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.8.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.10.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.12.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.14.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.16.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.18.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.20.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.22.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.24.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.26.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.28.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.30.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.32.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.34.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.36.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.38.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.40.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.42.indexer.attn_q_b.weight=f16", | |
| "--tensor-type", | |
| "blk.10.ffn_gate_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.10.ffn_up_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.10.ffn_down_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.14.ffn_gate_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.14.ffn_up_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.14.ffn_down_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.30.ffn_gate_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.30.ffn_up_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.30.ffn_down_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.34.ffn_gate_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.34.ffn_up_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.34.ffn_down_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.37.ffn_gate_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.37.ffn_up_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.37.ffn_down_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.38.ffn_gate_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.38.ffn_up_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.38.ffn_down_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.39.ffn_gate_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.39.ffn_up_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.39.ffn_down_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.40.ffn_gate_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.40.ffn_up_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.40.ffn_down_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.41.ffn_gate_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.41.ffn_up_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.41.ffn_down_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.42.ffn_gate_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.42.ffn_up_exps.weight=mxfp4", | |
| "--tensor-type", | |
| "blk.42.ffn_down_exps.weight=mxfp4", | |
| "--imatrix-strict", | |
| "--out", | |
| "DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128.gguf" | |
| ], | |
| [ | |
| "deepseek4-quantize", | |
| "--hf", | |
| "apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8", | |
| "--template", | |
| "antirez/deepseek-v4-gguf/ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0.header-only.sparse.gguf", | |
| "--dspark-support", | |
| "--experts", | |
| "iq2_xxs", | |
| "--routed-w2", | |
| "mxfp4", | |
| "--attention-proj", | |
| "q8_0", | |
| "--shared", | |
| "q8_0", | |
| "--dspark-block-size", | |
| "5", | |
| "--dspark-markov-rank", | |
| "256", | |
| "--dspark-noise-token-id", | |
| "128799", | |
| "--dspark-target-layers", | |
| "40,41,42", | |
| "--threads", | |
| "8", | |
| "--imatrix", | |
| "ox-ox/DeepSeek-V4-Flash-0731-GGUF/imatrix/DeepSeek-V4-Flash-0731-dspark-alias.dat", | |
| "--out", | |
| "DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128-DSpark-support.gguf", | |
| "--imatrix-strict" | |
| ] | |
| ], | |
| "appledouble_excluded": true | |
| } | |