Llama-3.1-8B-Instruct

This repository contains quantized versions of the model from the original repository: meta-llama/Llama-3.1-8B-Instruct.

Name	Quantization Method	Size (GB)
llama-3.1-8b-instruct.Q8_0.gguf	q8_0	7.95
llama-3.1-8b-instruct.Q4_0.gguf	q4_0	4.34

GGUF

Model size

8.03B params

Architecture

llama

4-bit

8-bit

Inference Providers NEW

This model is not currently available via any of the supported Inference Providers.

The model cannot be deployed to the HF Inference API: The model authors have turned it off explicitly.

Model tree for pbatra/Llama-3.1-8B-Instruct-GGUF

Base model

Finetuned

Quantized

(364)

this model