AlbertShi

AlbertShi's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes
• Articles

liked a model 3 days ago

SmallDoge/Doge-160M-Instruct

Question Answering • Updated 3 days ago • 31.7k • 8

upvoted a paper 25 days ago

Wonderful Matrices: Combining for a More Efficient and Effective Foundation Model Architecture

Paper • 2412.11834 • Published Dec 16, 2024 • 7

liked 4 models 25 days ago

SmallDoge/Doge-20M-checkpoint

Text Generation • Updated 6 days ago • 13.7k • 4

SmallDoge/Doge-60M-checkpoint

Text Generation • Updated 6 days ago • 13.8k • 3

SmallDoge/Doge-160M-checkpoint

Text Generation • Updated 6 days ago • 13.8k • 3

SmallDoge/Doge-20M-Instruct

Question Answering • Updated 6 days ago • 55.9k • 3

upvoted a collection 25 days ago

Doge

Collection

Doge family of small language models. • 8 items • Updated 1 day ago • 6

liked 3 models 25 days ago

SmallDoge/Doge-60M-Instruct

Question Answering • Updated 6 days ago • 64.3k • 5

SmallDoge/Doge-20M

Text Generation • Updated 6 days ago • 71.1k • 7

SmallDoge/Doge-60M

Text Generation • Updated 6 days ago • 74.7k • 4

reacted to JingzeShi's post with 👍🤯👀 25 days ago

view post

Post

2076

Only a single RTX 4090 running model pre-training is really slow, even for small language models!!! (https://huggingface.co/collections/JingzeShi/doge-slm-677fd879f8c4fd0f43e05458)

See translation

• 
• 

• 
• 
• 2 replies

·

👀

8

8

🤯

6

6

👍

4

4

+

Reply

reacted to JingzeShi's post with 🔥 25 days ago

view post

Post

1707

🤩warmup -> stable -> decay leanring rate scheduler:
😎use the Stable Phase CheckPoints to Continue Training the model on Any New Dataset without spikes of the training!!!
SmallDoge/Doge-20M-checkpoint
SmallDoge/Doge-60M-checkpoint

See translation

• 

• 
• 
• 
• 4 replies

·

🔥

7

7

😎

1

1

👀

1

1

🤗

1

1

+

Reply