--- license: llama3.1 language: - en base_model: - microsoft/phi-4 pipeline_tag: text-generation library_name: transformers tags: - text-generation-inference - llama3.1 - phi4 - LlamaForCausalLM - Corpus - trl model-index: - name: Megatron-Opus-14B-2.0 results: - task: type: text-generation name: Text Generation dataset: name: IFEval (0-Shot) type: wis-k/instruction-following-eval split: train args: num_few_shot: 0 metrics: - type: inst_level_strict_acc and prompt_level_strict_acc value: 66.94 name: averaged accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: BBH (3-Shot) type: SaylorTwift/bbh split: test args: num_few_shot: 3 metrics: - type: acc_norm value: 54.7 name: normalized accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MATH Lvl 5 (4-Shot) type: lighteval/MATH-Hard split: test args: num_few_shot: 4 metrics: - type: exact_match value: 27.79 name: exact match source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GPQA (0-shot) type: Idavidrein/gpqa split: train args: num_few_shot: 0 metrics: - type: acc_norm value: 14.54 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MuSR (0-shot) type: TAUR-Lab/MuSR args: num_few_shot: 0 metrics: - type: acc_norm value: 10.52 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU-PRO (5-shot) type: TIGER-Lab/MMLU-Pro config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 46.34 name: accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0 name: Open LLM Leaderboard --- ![xfghxxfdghfdgh.gif](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/dRGKgIqsV2h7W04Rp4Yld.gif) # **Megatron-Opus-14B-2.0 [ Exp ]** [Megatron-Opus-14B-2.0 ] Exp finetuned from Microsoft's Phi-4 is a state-of-the-art open model developed with a focus on responsible problem solving and advanced reasoning capabilities. Built upon a diverse blend of synthetic datasets, carefully filtered public domain websites, and high-quality academic books and Q&A datasets, Megatron-Opus-14B-2.0 ensures that small, capable models are trained with datasets of exceptional depth and precision. Megatron-Opus-14B-2.0 adopts a robust safety post-training approach using open-source and in-house synthetic datasets. This involves a combination of SFT (Supervised Fine-Tuning) and iterative DPO (Direct Preference Optimization) techniques, ensuring helpful and harmless outputs across various safety categories. # **Dataset Info** Megatron-Opus-14B-2.0 is fine-tuned on a carefully curated synthetic dataset generated using an advanced pipeline optimized for Chain of Thought (CoT) reasoning and Responsible Problem Breakdown (RPB) methodologies. This ensures that the model excels at: - **Logical reasoning** - **Step-by-step problem-solving** - **Breaking down complex tasks into manageable parts** The dataset also emphasizes responsible decision-making and fairness in generating solutions. # **Run with Transformers** ```python # pip install accelerate from transformers import AutoTokenizer, AutoModelForCausalLM import torch tokenizer = AutoTokenizer.from_pretrained("prithivMLmods/Megatron-Opus-14B-2.0") model = AutoModelForCausalLM.from_pretrained( "prithivMLmods/Megatron-Opus-14B-2.0", device_map="auto", torch_dtype=torch.bfloat16, ) input_text = "Explain the concept of black holes." input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") outputs = model.generate(**input_ids, max_new_tokens=64) print(tokenizer.decode(outputs[0])) ``` For chat-style interactions, use `tokenizer.apply_chat_template`: ```python messages = [ {"role": "user", "content": "Explain the concept of black holes."}, ] input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to("cuda") outputs = model.generate(**input_ids, max_new_tokens=256) print(tokenizer.decode(outputs[0])) ``` # **Intended Use** Megatron-Opus-14B-2.0 is tailored for a wide range of applications, especially those involving **advanced reasoning**, **multilingual capabilities**, and **responsible problem-solving**. Its primary use cases include: 1. **Responsible Problem Solving** - Breaking down complex problems into logical, actionable steps. - Offering ethical, well-rounded solutions in academic and professional contexts. 2. **Advanced Reasoning Tasks** - Excelling in mathematics, logic, and scientific reasoning. - Providing detailed explanations and systematic answers. 3. **Content Generation** - Assisting in generating high-quality content for various domains, including creative writing and technical documentation. - Supporting marketers, writers, and educators with detailed and well-structured outputs. 4. **Educational Support** - Acting as a virtual tutor for students by generating practice questions, answers, and detailed explanations. - Helping educators design learning material that promotes critical thinking and step-by-step problem-solving. 5. **Customer Support & Dialogue Systems** - Enabling chatbots and virtual assistants to provide accurate, helpful, and responsible responses. - Enhancing customer service with reasoning-driven automation. # **Limitations** Despite its strengths, Megatron-Opus-14B-2.0 has some limitations that users should be aware of: 1. **Bias and Fairness** - While great effort has been made to minimize biases, users should critically assess the model’s output in sensitive scenarios to avoid unintended bias. 2. **Contextual Interpretation** - The model may occasionally misinterpret highly nuanced prompts or ambiguous contexts, leading to suboptimal responses. 3. **Knowledge Cutoff** - Megatron-Opus-14B-2.0’s knowledge is static and based on the data available at the time of training. It does not include real-time updates or information on recent developments. 4. **Safety and Harmlessness** - Despite post-training safety alignment, inappropriate or harmful outputs may still occur. Continuous monitoring and human oversight are advised when using the model in critical contexts. 5. **Computational Requirements** - Deploying Megatron-Opus-14B-2.0 efficiently may require substantial computational resources, particularly for large-scale deployments or real-time applications. # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/prithivMLmods__Megatron-Opus-14B-2.0-details)! Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=prithivMLmods%2FMegatron-Opus-14B-2.0&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)! | Metric |Value (%)| |-------------------|--------:| |**Average** | 36.80| |IFEval (0-Shot) | 66.94| |BBH (3-Shot) | 54.70| |MATH Lvl 5 (4-Shot)| 27.79| |GPQA (0-shot) | 14.54| |MuSR (0-shot) | 10.52| |MMLU-PRO (5-shot) | 46.34|