metadata
license: apache-2.0
tags:
- generated_from_trainer
- storytelling
- fiction
- tiny-stories
pipeline_tag: text-generation
library_name: transformers
Athspi LLM
🧠 A small but capable language model for creative story generation, trained on the TinyStories dataset.
Model Details
Architecture
- Model Type: Transformer-based language model
- Layers: 4
- Embedding Dim: 384
- Heads: 6
- Sequence Length: 128 tokens
- Parameters: ~28M
Training Data
- Dataset: TinyStories
- Training Coverage: 5% of dataset (~100k samples)
Usage
Installation
pip install torch transformers sentencepiece