hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4
Text Generation
•
Updated
•
3.75k
•
37
Llama 3.1 GPTQ, AWQ, and BNB Quants
Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗
-
-
hugging-quants/Meta-Llama-3.1-405B-Instruct-BNB-NF4
Text Generation • Updated • 183 • 5 -
hugging-quants/Meta-Llama-3.1-405B-Instruct-GPTQ-INT4
Text Generation • Updated • 677 • 16 -
hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4
Text Generation • Updated • 261k • 97 -
unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit
Text Generation • Updated • 8.36k • 29 -
hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4
Text Generation • Updated • 4.76k • 22 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4
Text Generation • Updated • 566k • 62 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4
Text Generation • Updated • 463 • 8 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4
Text Generation • Updated • 11.1k • 22