Hymba: A Hybrid-head Architecture for Small Language Models
Paper
•
2411.13676
•
Published
•
42
James PRO
jtatman
AI & ML interests
improving domain specific models and re-sampling data, refining datasets for use in different modalities, small scale micro-llm clusters using quantized and smoothed models, and all emerging llm stack connecting technologies. Small models rock.
Recent Activity
liked
a model
about 10 hours ago
NousResearch/DeepHermes-3-Llama-3-8B-Preview
liked
a model
about 11 hours ago
HuggingFaceTB/SmolVLM-256M-Instruct
liked
a model
about 13 hours ago
BAAI/bge-base-en-v1.5
Organizations
jtatman's activity
upvoted
a
paper
3 months ago
upvoted
a
paper
6 months ago
upvoted
a
collection
7 months ago
upvoted
an
article
7 months ago
Article
Llama 3.1 - 405B, 70B & 8B with multilinguality and long context
•
227
upvoted
a
paper
7 months ago
upvoted
a
paper
8 months ago
upvoted
an
article
8 months ago
Article
Welcome Gemma - Google's new open LLM
•
22
upvoted
a
collection
9 months ago
upvoted
an
article
10 months ago
Article
SeeMoE: Implementing a MoE Vision Language Model from Scratch
By
•
•
34
upvoted
a
paper
12 months ago
upvoted
4
papers
about 1 year ago
Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding
Paper
•
2401.12954
•
Published
•
30
Transformers are Multi-State RNNs
Paper
•
2401.06104
•
Published
•
37
LLM in a flash: Efficient Large Language Model Inference with Limited Memory
Paper
•
2312.11514
•
Published
•
259
Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers
Paper
•
2311.10642
•
Published
•
26