Mamba-2.8b-slimpj is a model using the Mamba architecture, with 2.8B parameters, trained for 600B tokens on the SlimPajama dataset.
Model code: https://github.com/state-spaces/mamba/tree/main
To load the model, follow the installation instruction in the code repo, and then:
from mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel
model = MambaLMHeadModel.from_pretrained("state-spaces/mamba-2.8b-slimpj")
- Downloads last month
- 2,097
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no pipeline_tag.