This repository contains the 300BT preview of the Open LLaMA model based on the RedPajama 1T dataset, converted to GGML and quantized to 4 bit, ready to be used with llama.cpp
Usage
In order to use this model with llama.cpp
- install llama.cpp as described in the docs
- download this model
- move it into the
modelssubfolder of llama.cpp - run inferences with the additional parameter
-m ./models/7B/ggml-openllama-7b-300bt-q4_0.bin
For example,
./main -m ./models/7B/ggml-openllama-7b-300bt-q4_0.bin -n 128 -p "who was Joseph Weizenbaum?"
License
Since the underlying model is itself Apache 2.0 licensed, this repo is as well.
license: apache-2.0
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.