Text Generation

Artifacts for the GoldFinch paper

Code available at https://github.com/recursal/GoldFinch-paper

Contains checkpoints for various small scale ablations and 1.5B scale models, all trained on minipile

Bibtex citation

@misc{goldstein2024goldfinchhighperformancerwkvtransformer,
      title={GoldFinch: High Performance RWKV/Transformer Hybrid with Linear Pre-Fill and Extreme KV-Cache Compression}, 
      author={Daniel Goldstein and Fares Obeid and Eric Alcaide and Guangyu Song and Eugene Cheah},
      year={2024},
      eprint={2407.12077},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2407.12077}, 
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.