Hugging Face's logo

minyichen
/
Llama-3-Taiwan-8B-Instruct-GPTQ

Text Generation
Transformers
Safetensors
Chinese
English
llama
llama-3
conversational
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community

Llama-3-Taiwan-8B-Instruct - GPTQ

Description

This repo contains GPTQ model files for Llama-3-Taiwan-8B-Instruct.

Quantization parameter

  • Bits : 4
  • Group Size : 128
  • Act Order : Yes
  • Damp % : 0.1
  • Seq Len : 2048
  • Size : 5.34 GB

It tooks about 50 mins to quantize on H100.

Downloads last month
85
Safetensors
Model size
1.99B params
Tensor type
I32
·
BF16
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported Inference Providers.

Model tree for minyichen/Llama-3-Taiwan-8B-Instruct-GPTQ

Base model

meta-llama/Meta-Llama-3-70B
Finetuned
yentinglin/Llama-3-Taiwan-8B-Instruct
Quantized
(16)
this model

Collection including minyichen/Llama-3-Taiwan-8B-Instruct-GPTQ