Here are a few GGUF(v2) quantizations of the model pankajmathur/orca_mini_3b

GGUF

Model size

3.43B params

Architecture

llama

4-bit

5-bit

Inference Providers NEW

This model is not currently available via any of the supported Inference Providers.

The model cannot be deployed to the HF Inference API: The model has no library tag.

Aryanne
/

Orca-Mini-3B-gguf

Datasets used to train Aryanne/Orca-Mini-3B-gguf