1.36k
The Ultra-Scale Playbook
š
The ultimate guide to training LLM on large GPU Clusters
Alberto Cetoli PRO
fractalego
AI & ML interests
Entity/relation extraction, Q&A, Summarisation
Recent Activity
liked
a Space
4 days ago
nanotron/ultrascale-playbook
liked
a model
7 days ago
NousResearch/DeepHermes-3-Llama-3-8B-Preview
reacted
to
csabakecskemeti's
post
with š
7 days ago
I found if we apply the reasoning system prompt (that has been published on the https://huggingface.co/NousResearch/DeepHermes-3-Llama-3-8B-Preview model card) other models are also react to it and start mimicking reasoning. Some better some worse. I've seen internal monologue and self questioning.
Here's a blogpost about it:
http://devquasar.com/ai/reasoning-system-prompt/
Organizations
fractalego's activity
liked
a
Space
4 days ago
liked
a
model
7 days ago
reacted to
csabakecskemeti's
post with š
7 days ago
Post
1583
I found if we apply the reasoning system prompt (that has been published on the
NousResearch/DeepHermes-3-Llama-3-8B-Preview model card) other models are also react to it and start mimicking reasoning. Some better some worse. I've seen internal monologue and self questioning.
Here's a blogpost about it:
http://devquasar.com/ai/reasoning-system-prompt/
Here's a blogpost about it:
http://devquasar.com/ai/reasoning-system-prompt/
reacted to
mmhamdy's
post with š
9 days ago
Post
2952
ā Evaluating Long Context #2: SCROLLS and ZeroSCROLLS
In this series of posts about tracing the history of long context evaluation, we started with Long Range Arena (LRA). Introduced in 2020, Long Range Arens (LRA) is one of the earliest benchmarks designed to tackle the challenge of long context evaluation. But it wasn't introduced to evaluate LLMs, but rather the transformer architecture in general.
š The SCROLLS benchmark, introduced in 2022, addresses this gap in NLP/LLM research. SCROLLS challenges models with tasks that require reasoning over extended sequences (according to 2022 standards). So, what does it offer?
1ļøā£ Long Text Focus: SCROLLS (unlike LRA) focus mainly on text and contain inputs with thousands of words, testing models' ability to synthesize information across lengthy documents.
2ļøā£ Diverse Tasks: Includes summarization, question answering, and natural language inference across domains like literature, science, and business.
3ļøā£ Unified Format: All datasets are available in a text-to-text format, facilitating easy evaluation and comparison of models.
Building on SCROLLS, ZeroSCROLLS takes long text evaluation to the next level by focusing on zero-shot learning. Other features include:
1ļøā£ New Tasks: Introduces tasks like sentiment aggregation and sorting book chapter summaries.
2ļøā£ Leaderboard: A live leaderboard encourages continuous improvement and competition among researchers.
š” What are some other landmark benchmarks in the history of long context evaluation? Feel free to share your thoughts and suggestions in the comments.
- SCROLLS Paper: SCROLLS: Standardized CompaRison Over Long Language Sequences (2201.03533)
- ZeroSCROLLS Paper: ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding (2305.14196)
In this series of posts about tracing the history of long context evaluation, we started with Long Range Arena (LRA). Introduced in 2020, Long Range Arens (LRA) is one of the earliest benchmarks designed to tackle the challenge of long context evaluation. But it wasn't introduced to evaluate LLMs, but rather the transformer architecture in general.
š The SCROLLS benchmark, introduced in 2022, addresses this gap in NLP/LLM research. SCROLLS challenges models with tasks that require reasoning over extended sequences (according to 2022 standards). So, what does it offer?
1ļøā£ Long Text Focus: SCROLLS (unlike LRA) focus mainly on text and contain inputs with thousands of words, testing models' ability to synthesize information across lengthy documents.
2ļøā£ Diverse Tasks: Includes summarization, question answering, and natural language inference across domains like literature, science, and business.
3ļøā£ Unified Format: All datasets are available in a text-to-text format, facilitating easy evaluation and comparison of models.
Building on SCROLLS, ZeroSCROLLS takes long text evaluation to the next level by focusing on zero-shot learning. Other features include:
1ļøā£ New Tasks: Introduces tasks like sentiment aggregation and sorting book chapter summaries.
2ļøā£ Leaderboard: A live leaderboard encourages continuous improvement and competition among researchers.
š” What are some other landmark benchmarks in the history of long context evaluation? Feel free to share your thoughts and suggestions in the comments.
- SCROLLS Paper: SCROLLS: Standardized CompaRison Over Long Language Sequences (2201.03533)
- ZeroSCROLLS Paper: ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding (2305.14196)
liked
a
dataset
10 days ago
liked
a
model
13 days ago
liked
a
model
21 days ago
upvoted
an
article
21 days ago
Article
Open-R1: Update #1
By
and 7 others
ā¢
ā¢
286
liked
a
model
22 days ago
upvoted
an
article
24 days ago
Article
ColPali: Efficient Document Retrieval with Vision Language Models š
By
ā¢
ā¢
205
upvoted
a
paper
27 days ago
upvoted
a
paper
about 1 month ago
upvoted
an
article
about 1 month ago
Article
Visualize and understand GPU memory in PyTorch
ā¢
187
liked
a
dataset
about 1 month ago
liked
a
model
about 2 months ago
reacted to
mitkox's
post with š¤Æš„ā
about 2 months ago
Post
2480
Can it run DeepSeek V3 671B is the new 'can it run Doom'.
How minimalistic can I go with on device AI with behemoth models - here I'm running DeepSeek V3 MoE on a single A6000 GPU.
Not great, not terrible, for this minimalistic setup. I love the Mixture of Experts architectures. Typically I'm running my core LLM distributed over the 4 GPUs.
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
How minimalistic can I go with on device AI with behemoth models - here I'm running DeepSeek V3 MoE on a single A6000 GPU.
Not great, not terrible, for this minimalistic setup. I love the Mixture of Experts architectures. Typically I'm running my core LLM distributed over the 4 GPUs.
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
- 5 replies
liked
2
datasets
about 2 months ago