File size: 8,570 Bytes
8b3dfe9 9d4d983 c3ddba5 3a9b6ea b0b9e5b a928116 aa8fcaa a928116 b0b9e5b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 |
---
license: llama3.1
language:
- en
base_model:
- microsoft/phi-4
pipeline_tag: text-generation
library_name: transformers
tags:
- text-generation-inference
- llama3.1
- phi4
- LlamaForCausalLM
- Corpus
- trl
model-index:
- name: Megatron-Opus-14B-2.0
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: IFEval (0-Shot)
type: wis-k/instruction-following-eval
split: train
args:
num_few_shot: 0
metrics:
- type: inst_level_strict_acc and prompt_level_strict_acc
value: 66.94
name: averaged accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: BBH (3-Shot)
type: SaylorTwift/bbh
split: test
args:
num_few_shot: 3
metrics:
- type: acc_norm
value: 54.7
name: normalized accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MATH Lvl 5 (4-Shot)
type: lighteval/MATH-Hard
split: test
args:
num_few_shot: 4
metrics:
- type: exact_match
value: 27.79
name: exact match
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GPQA (0-shot)
type: Idavidrein/gpqa
split: train
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 14.54
name: acc_norm
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MuSR (0-shot)
type: TAUR-Lab/MuSR
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 10.52
name: acc_norm
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU-PRO (5-shot)
type: TIGER-Lab/MMLU-Pro
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 46.34
name: accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
name: Open LLM Leaderboard
---
# **Megatron-Opus-14B-2.0 [ Exp ]**
[Megatron-Opus-14B-2.0 ] Exp finetuned from Microsoft's Phi-4 is a state-of-the-art open model developed with a focus on responsible problem solving and advanced reasoning capabilities. Built upon a diverse blend of synthetic datasets, carefully filtered public domain websites, and high-quality academic books and Q&A datasets, Megatron-Opus-14B-2.0 ensures that small, capable models are trained with datasets of exceptional depth and precision.
Megatron-Opus-14B-2.0 adopts a robust safety post-training approach using open-source and in-house synthetic datasets. This involves a combination of SFT (Supervised Fine-Tuning) and iterative DPO (Direct Preference Optimization) techniques, ensuring helpful and harmless outputs across various safety categories.
# **Dataset Info**
Megatron-Opus-14B-2.0 is fine-tuned on a carefully curated synthetic dataset generated using an advanced pipeline optimized for Chain of Thought (CoT) reasoning and Responsible Problem Breakdown (RPB) methodologies. This ensures that the model excels at:
- **Logical reasoning**
- **Step-by-step problem-solving**
- **Breaking down complex tasks into manageable parts**
The dataset also emphasizes responsible decision-making and fairness in generating solutions.
# **Run with Transformers**
```python
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("prithivMLmods/Megatron-Opus-14B-2.0")
model = AutoModelForCausalLM.from_pretrained(
"prithivMLmods/Megatron-Opus-14B-2.0",
device_map="auto",
torch_dtype=torch.bfloat16,
)
input_text = "Explain the concept of black holes."
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
outputs = model.generate(**input_ids, max_new_tokens=64)
print(tokenizer.decode(outputs[0]))
```
For chat-style interactions, use `tokenizer.apply_chat_template`:
```python
messages = [
{"role": "user", "content": "Explain the concept of black holes."},
]
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to("cuda")
outputs = model.generate(**input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0]))
```
# **Intended Use**
Megatron-Opus-14B-2.0 is tailored for a wide range of applications, especially those involving **advanced reasoning**, **multilingual capabilities**, and **responsible problem-solving**. Its primary use cases include:
1. **Responsible Problem Solving**
- Breaking down complex problems into logical, actionable steps.
- Offering ethical, well-rounded solutions in academic and professional contexts.
2. **Advanced Reasoning Tasks**
- Excelling in mathematics, logic, and scientific reasoning.
- Providing detailed explanations and systematic answers.
3. **Content Generation**
- Assisting in generating high-quality content for various domains, including creative writing and technical documentation.
- Supporting marketers, writers, and educators with detailed and well-structured outputs.
4. **Educational Support**
- Acting as a virtual tutor for students by generating practice questions, answers, and detailed explanations.
- Helping educators design learning material that promotes critical thinking and step-by-step problem-solving.
5. **Customer Support & Dialogue Systems**
- Enabling chatbots and virtual assistants to provide accurate, helpful, and responsible responses.
- Enhancing customer service with reasoning-driven automation.
# **Limitations**
Despite its strengths, Megatron-Opus-14B-2.0 has some limitations that users should be aware of:
1. **Bias and Fairness**
- While great effort has been made to minimize biases, users should critically assess the model’s output in sensitive scenarios to avoid unintended bias.
2. **Contextual Interpretation**
- The model may occasionally misinterpret highly nuanced prompts or ambiguous contexts, leading to suboptimal responses.
3. **Knowledge Cutoff**
- Megatron-Opus-14B-2.0’s knowledge is static and based on the data available at the time of training. It does not include real-time updates or information on recent developments.
4. **Safety and Harmlessness**
- Despite post-training safety alignment, inappropriate or harmful outputs may still occur. Continuous monitoring and human oversight are advised when using the model in critical contexts.
5. **Computational Requirements**
- Deploying Megatron-Opus-14B-2.0 efficiently may require substantial computational resources, particularly for large-scale deployments or real-time applications.
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/prithivMLmods__Megatron-Opus-14B-2.0-details)!
Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=prithivMLmods%2FMegatron-Opus-14B-2.0&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
| Metric |Value (%)|
|-------------------|--------:|
|**Average** | 36.80|
|IFEval (0-Shot) | 66.94|
|BBH (3-Shot) | 54.70|
|MATH Lvl 5 (4-Shot)| 27.79|
|GPQA (0-shot) | 14.54|
|MuSR (0-shot) | 10.52|
|MMLU-PRO (5-shot) | 46.34|
|