Improve model card for LoRI-D_code_llama3_rank_32: Add details, usage, and license

README.md

@@ -2,192 +2,203 @@
 base_model: meta-llama/Meta-Llama-3-8B
 library_name: peft
 pipeline_tag: text-generation
+license: apache-2.0
+tags:
+  - lora
+  - peft
+  - fine-tuning
+  - language-model
+  - code-generation
+  - natural-language-understanding
+  - mathematical-reasoning
+  - safety-alignment
+  - multi-task
+  - continual-learning
+  - llama
+  - llama-3
 ---
 
 # Model Card for LoRI-D_code_llama3_rank_32
 
-This model is part of [LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation](https://arxiv.org/abs/2504.07448).
-
-<!-- Provide a quick summary of what the model is/does. -->
+This model is part of [LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation](https://huggingface.co/papers/2504.07448).
 
+LoRI (LoRA with Reduced Interference) is a simple yet effective approach that freezes the projection matrices $A$ as random projections and sparsifies the matrices $B$ using task-specific masks. This design substantially reduces the number of trainable parameters while maintaining strong task performance. Moreover, LoRI minimizes cross-task interference in adapter merging by leveraging the orthogonality between adapter subspaces, and supports continual learning by using sparsity to mitigate catastrophic forgetting. Extensive experiments across natural language understanding, mathematical reasoning, code generation, and safety alignment tasks demonstrate that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than LoRA. In multi-task experiments, LoRI enables effective adapter merging and continual learning with reduced cross-task interference.
 
+<div align="center">
+    <img src="https://github.com/juzhengz/LoRI/raw/main/LoRI.png" alt="LoRI" width="80%">
+</div>
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
-
-
+LoRI (LoRA with Reduced Interference) is a novel parameter-efficient fine-tuning (PEFT) method designed to address the overhead and parameter interference often encountered in multi-task scenarios for Large Language Models (LLMs). By freezing projection matrices `A` as random projections and sparsifying matrices `B` with task-specific masks, LoRI significantly reduces trainable parameters while preserving strong task performance and minimizing cross-task interference. This approach also supports continual learning by leveraging sparsity to mitigate catastrophic forgetting.
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+This specific model, `LoRI-D_code_llama3_rank_32`, is a LoRI adapter based on the `meta-llama/Meta-Llama-3-8B` model, specifically fine-tuned for code generation tasks.
 
-### Model Sources [optional]
+-   **Developed by:** Juzheng Zhang, Jiacheng You, Ashwinee Panda, Tom Goldstein
+-   **Model type:** Parameter-Efficient Fine-Tuning (PEFT) adapter; a variant of Low-Rank Adaptation (LoRA).
+-   **Language(s) (NLP):** English (as the primary language for the base model and tasks such as NLU, mathematical reasoning, code generation, and safety alignment).
+-   **License:** Apache-2.0
+-   **Finetuned from model:** `meta-llama/Meta-Llama-3-8B`
 
-<!-- Provide the basic links for the model. -->
+### Model Sources
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+-   **Repository:** [https://github.com/juzhengz/LoRI/](https://github.com/juzhengz/LoRI/)
+-   **Paper:** [https://huggingface.co/papers/2504.07448](https://huggingface.co/papers/2504.07448)
+-   **Hugging Face Collection:** [https://huggingface.co/collections/tomg-group-umd/lori-adapters-67f795549d792613e1290011](https://huggingface.co/collections/tomg-group-umd/lori-adapters-67f795549d792613e1290011)
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
+LoRI adapters are intended for efficient fine-tuning of Large Language Models on various downstream tasks, including:
+-   Natural Language Understanding (NLU)
+-   Mathematical Reasoning
+-   Code Generation
+-   Safety Alignment
 
-### Downstream Use [optional]
+They can be loaded as PEFT adapters on top of a base LLM to enhance performance with reduced parameter overhead.
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+### Downstream Use
 
-[More Information Needed]
+LoRI enables effective adapter merging for multi-task applications, allowing a single model to perform well across several distinct tasks without significant performance degradation. It also supports continual learning, facilitating model adaptation to new tasks sequentially while mitigating catastrophic forgetting.
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+This model is not intended for generating harmful, biased, or unethical content. Users should ensure compliance with the ethical guidelines of the base models and apply appropriate content filtering. It is not suitable for standalone use without a compatible base LLM or for tasks outside its trained domains without further adaptation.
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
+As an adapter for Large Language Models, LoRI inherits potential biases and limitations from its base model (`meta-llama/Meta-Llama-3-8B`) and the datasets it was fine-tuned on. Users should be aware that results might reflect biases present in the training data. While LoRI aims to reduce cross-task interference, complete elimination of such interference or catastrophic forgetting is not guaranteed. Performance on highly out-of-distribution tasks may be limited.
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. It is recommended to perform further evaluation on specific target datasets and contexts to understand its performance and potential biases relevant to the application. Implement appropriate safeguards and content filtering when deploying the model in production environments.
 
 ## How to Get Started with the Model
 
 Use the code below to get started with the model.
 
-[More Information Needed]
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+# Load the base model
+base_model_path = "meta-llama/Meta-Llama-3-8B"
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_path,
+    torch_dtype=torch.bfloat16, # Use torch.float16 if bfloat16 is not supported
+    device_map="auto" # Distributes the model across available devices
+)
+
+# Load the LoRI adapter (this model is for code generation)
+lori_adapter_path = "tomg-group-umd/LoRI-D_code_llama3_rank_32"
+model = PeftModel.from_pretrained(base_model, lori_adapter_path)
+
+# Load the tokenizer
+tokenizer = AutoTokenizer.from_pretrained(base_model_path)
+
+# Example usage for code generation
+prompt = "def factorial(n):
+    if n == 0:
+        return 1
+    else:
+        return n * "
+
+# Apply chat template if using a chat-tuned base model (Llama-3 is chat-tuned)
+# Or directly tokenize if the base model is not primarily chat-tuned
+messages = [{"role": "user", "content": prompt}]
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+# Generate text
+outputs = model.generate(
+    input_ids,
+    max_new_tokens=50,
+    do_sample=True,
+    temperature=0.7,
+    top_p=0.9
+)
+
+# Decode and print the generated response (excluding the prompt for cleaner output)
+response = tokenizer.decode(outputs[0][len(input_ids[0]):], skip_special_tokens=True)
+print(response)
+```
 
 ## Training Details
 
+LoRI is implemented using Fully Sharded Data Parallel (FSDP) and can be executed in a multi-GPU environment. Training scripts are provided for various tasks covering Natural Language Understanding (NLU), Code Generation, Mathematical Reasoning, and Safety Alignment.
+
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+LoRI adapters were trained on datasets corresponding to various tasks:
+-   **Code generation tasks:** CodeAlpaca
+-   **Mathematical reasoning tasks:** GSM8K
+-   **Safety alignment tasks:** Saferpaca
+-   **Natural Language Understanding (NLU) tasks:** Specific NLU datasets (refer to the paper for a full list).
 
-[More Information Needed]
+More details on these datasets can be found in the [LoRI GitHub repository](https://github.com/juzhengz/LoRI/).
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
+LoRI training typically involves two main stages:
+1.  **LoRI-D (Decomposition):** Initial training where projection matrices `A` are frozen as random projections, and matrices `B` are trained. This stage is used to extract sparse masks.
+2.  **LoRI-S (Sparsification):** Continues training from LoRI-D, where matrices `B` are sparsified using task-specific masks (e.g., 90% sparsity for `LoRI-S`).
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+-   **PEFT Type:** LORA
+-   **LoRA Alpha (`lora_alpha`):** 64
+-   **LoRA Rank (`r`):** 32 (for this specific model `LoRI-D_code_llama3_rank_32`)
+-   **LoRA Dropout (`lora_dropout`):** 0.05
+-   **Target Modules:** `q_proj`, `o_proj`, `k_proj`, `v_proj`, `down_proj`, `gate_proj`, `up_proj`
+-   **Training Regime:** Mixed precision (e.g., bfloat16 commonly used with Llama-3).
+-   **Optimization:** Fully Sharded Data Parallel (FSDP)
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
+Extensive experiments were conducted across natural language understanding, mathematical reasoning, code generation, and safety alignment tasks. The paper demonstrates that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than standard LoRA. In multi-task experiments, LoRI enables effective adapter merging and continual learning with reduced cross-task interference. Code generation performance on HumanEval is evaluated using the `bigcode-evaluation-harness`. For detailed quantitative results, please refer to the [paper](https://huggingface.co/papers/2504.07448).
 
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Technical Specifications [optional]
+## Technical Specifications
 
 ### Model Architecture and Objective
 
-[More Information Needed]
+LoRI introduces a novel architecture built upon existing Transformer-based LLMs, adapting the standard LoRA approach. It achieves efficiency and interference reduction by:
+-   Freezing the low-rank projection matrix `A` as random projections.
+-   Sparsifying the low-rank projection matrix `B` using task-specific masks.
+The primary objective is to minimize cross-task interference in multi-task low-rank adaptation, while significantly reducing the trainable parameter count and maintaining strong performance.
 
 ### Compute Infrastructure
 
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
+The training and evaluation were performed on multi-GPU environments, leveraging technologies like FSDP. Experiments were conducted with LLaMA-3-8B and Mistral-7B base models.
 
 #### Software
 
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
+The implementation relies on `PyTorch`, `transformers`, and `peft` libraries. It builds on the codebase of [dpo-rlaif](https://github.com/architsharma97/dpo-rlaif) and incorporates code from [lottery-ticket-adaptation](https://github.com/kiddyboots216/lottery-ticket-adaptation). Evaluation for code generation uses the [bigcode-evaluation-harness](https://github.com/bigcode-project/bigcode-evaluation-harness).
 
-## More Information [optional]
+## Citation
 
-[More Information Needed]
+If you use LoRI in your work, please cite:
 
-## Model Card Authors [optional]
+```bibtex
+@article{zhang2025lori,
+  title={LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation},
+  author={Zhang, Juzheng and You, Jiacheng and Panda, Ashwinee and Goldstein, Tom},
+  journal={arXiv preprint arXiv:2504.07448},
+  year={2025}
+}
+```
 
-[More Information Needed]
+## More Information
 
-## Model Card Contact
+For further details on installation, training from scratch, adapter merging, continual learning, and customizing base models/losses, please refer to the [official GitHub repository](https://github.com/juzhengz/LoRI/).
 
-[More Information Needed]
 ### Framework versions
 
 - PEFT 0.12.0