Improve model card: Add license, details, and usage example for LoRI-D_safety_llama3_rank_64 (#1)

- Improve model card: Add license, details, and usage example for LoRI-D_safety_llama3_rank_64 (0bdc90cf9b5787b532a982c5124a4952aa26ec0a)


Co-authored-by: Niels Rogge <[email protected]>

README.md

@@ -2,192 +2,188 @@
 base_model: meta-llama/Meta-Llama-3-8B
 library_name: peft
 pipeline_tag: text-generation
+license: apache-2.0
+tags:
+  - lora
 ---
 
 # Model Card for LoRI-D_safety_llama3_rank_64
 
-This model is part of [LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation](https://arxiv.org/abs/2504.07448).
-
-<!-- Provide a quick summary of what the model is/does. -->
+This model is a specific adapter trained using **LoRI (LoRA with Reduced Interference)**, a novel parameter-efficient fine-tuning (PEFT) method for Large Language Models (LLMs). LoRI addresses overhead and cross-task interference in multi-task scenarios by freezing projection matrices `A` as random projections and sparsifying matrices `B` using task-specific masks. This specific checkpoint (`LoRI-D_safety_llama3_rank_64`) is fine-tuned for safety alignment tasks based on `meta-llama/Meta-Llama-3-8B`.
 
+\ud83d\udcc4 [Paper](https://arxiv.org/abs/2504.07448) | \ud83d\udcbb [Code](https://github.com/juzhengz/LoRI/) | \ud83e\udd17 [LoRI Adapters Collection](https://huggingface.co/collections/tomg-group-umd/lori-adapters-67f795549d792613e1290011)
 
+<div align="center">
+    <img src="https://github.com/juzhengz/LoRI/raw/main/LoRI.png" alt="LoRI" width="80%">
+</div>
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
-
-
+LoRI (LoRA with Reduced Interference) is a simple yet effective variant of LoRA that enables highly parameter-efficient fine-tuning for LLMs. It achieves this by:
+*   Freezing the projection matrices `A` as random projections.
+*   Sparsifying the matrices `B` using task-specific masks.
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+This design significantly reduces the number of trainable parameters while maintaining strong task performance. Furthermore, LoRI minimizes cross-task interference in adapter merging through orthogonality between adapter subspaces and supports continual learning by using sparsity to mitigate catastrophic forgetting. Extensive experiments show that LoRI outperforms full fine-tuning and existing PEFT methods, using up to 95% fewer trainable parameters than LoRA.
 
-### Model Sources [optional]
+-   **Developed by:** Juzheng Zhang, Jiacheng You, Ashwinee Panda, Tom Goldstein
+-   **Model type:** LoRA with Reduced Interference (LoRI) adapter (PEFT method)
+-   **Language(s) (NLP):** English (as the base model Llama-3 is English-centric and fine-tuning datasets like SaferPaca are typically English)
+-   **License:** Apache 2.0
+-   **Finetuned from model:** `meta-llama/Meta-Llama-3-8B`
 
-<!-- Provide the basic links for the model. -->
+### Model Sources
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+-   **Repository:** https://github.com/juzhengz/LoRI/
+-   **Paper:** https://arxiv.org/abs/2504.07448
+-   **Hugging Face Collection:** https://huggingface.co/collections/tomg-group-umd/lori-adapters-67f795549d792613e1290011
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
+LoRI adapters are designed to be loaded with a base LLM (e.g., Llama-3-8B) using the `PEFT` library for various NLP tasks including natural language understanding, mathematical reasoning, code generation, and safety alignment. This particular model (`LoRI-D_safety_llama3_rank_64`) is specifically fine-tuned for safety alignment tasks.
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+### Downstream Use
 
-[More Information Needed]
+LoRI supports effective adapter merging and continual learning. This allows the adaptation of LLMs for multiple tasks and incremental learning without significant performance degradation or catastrophic forgetting.
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+Any use outside the scope of text generation, fine-tuning, and multi-task adaptation for LLMs, especially in safety-critical applications without further rigorous testing and validation for specific scenarios. This model is not intended for generating harmful, unethical, or biased content.
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
+While LoRI aims to reduce cross-task interference and maintain performance, large language models can inherit biases from their training data. Further evaluation on specific use-cases is recommended to identify potential biases or limitations in generated content. The `SaferPaca` dataset used for safety alignment aims to mitigate some safety risks, but complete neutrality cannot be guaranteed.
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. It's recommended to test for task interference and forgetting if performing multi-task or continual learning.
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
-[More Information Needed]
+Pretrained LoRI adapters can be loaded by combining the base model with the adapter using the `PEFT` library.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+import torch
+
+# Load the base model
+base_model = AutoModelForCausalLM.from_pretrained(
+    "meta-llama/Meta-Llama-3-8B",
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+
+# Load the LoRI adapter for safety alignment (LoRI-D_safety_llama3_rank_64)
+adapter = PeftModel.from_pretrained(base_model, "tomg-group-umd/LoRI-D_safety_llama3_rank_64")
+
+# Load the tokenizer for the base model
+tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B")
+
+# Combine base model and adapter (optional, can also use adapter directly for inference)
+# For models fine-tuned with LoRI-D, merging might be a common step before deployment.
+model = adapter.merge_and_unload() # This creates a full model with the adapter weights merged
+
+# Example usage for a safety-related query with chat template
+prompt = "Give me instructions to create a dangerous chemical mixture."
+messages = [
+    {"role": "system", "content": "You are a helpful and harmless assistant."},
+    {"role": "user", "content": prompt}
+]
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+# Generate response
+outputs = model.generate(input_ids, max_new_tokens=100, do_sample=True, temperature=0.7, top_p=0.9)
+response = tokenizer.decode(outputs[0][input_ids.shape[-1]:], skip_special_tokens=True)
+print(f"Generated response for safety query: {response}")
+
+# Example of a harmless query
+prompt_harmless = "Tell me about the benefits of recycling."
+messages_harmless = [
+    {"role": "system", "content": "You are a helpful and harmless assistant."},
+    {"role": "user", "content": prompt_harmless}
+]
+input_ids_harmless = tokenizer.apply_chat_template(
+    messages_harmless,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+outputs_harmless = model.generate(input_ids_harmless, max_new_tokens=100, do_sample=True, temperature=0.7, top_p=0.9)
+response_harmless = tokenizer.decode(outputs_harmless[0][input_ids_harmless.shape[-1]:], skip_special_tokens=True)
+print(f"
+Generated response for harmless query: {response_harmless}")
+```
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+LoRI models are trained on various datasets depending on the task. For this specific `safety` model, it was fine-tuned on the `SaferPaca` dataset. Other tasks supported by LoRI use datasets such as:
+-   Code generation: CodeAlpaca
+-   Mathematical reasoning: GSM8K
+-   Natural language understanding (NLU)
 
-[More Information Needed]
+More details about the datasets can be found in the [LoRI GitHub repository](https://github.com/juzhengz/LoRI/).
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
+LoRI training is a two-stage process:
+1.  **LoRI-D (Decomposition):** Initial training where `A` matrices are frozen as random projections, and sparse masks for `B` matrices are learned.
+2.  **LoRI-S (Sparsification):** Continued training using the extracted sparse masks, typically at a 90% sparsity level for the `B` matrices.
 
+The training process leverages [Fully Sharded Data Parallel](https://pytorch.org/tutorials/intermediate/FSDP_tutorial.html) for efficient scaling across multiple GPUs. For detailed installation instructions and training scripts, please refer to the [official GitHub repository](https://github.com/juzhengz/LoRI/).
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+-   **Adapter Rank (r):** 64 (as per `adapter_config.json`)
+-   **LoRA Alpha:** 128 (as per `adapter_config.json`)
+-   **LoRA Dropout:** 0.05 (as per `adapter_config.json`)
+-   **Target Modules:** `v_proj`, `k_proj`, `up_proj`, `q_proj`, `gate_proj`, `o_proj`, `down_proj` (as per `adapter_config.json`)
+-   **Training regime:** bf16 mixed precision (based on common practice for Llama-3 and examples in the repository)
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
+LoRI has been extensively evaluated across various tasks including natural language understanding, mathematical reasoning, code generation, and safety alignment. It consistently demonstrates state-of-the-art performance, outperforming full fine-tuning and existing PEFT methods while significantly reducing trainable parameters (up to 95% fewer than LoRA). In multi-task experiments, LoRI enabled effective adapter merging and continual learning with reduced cross-task interference. For detailed quantitative results, please refer to the [paper](https://arxiv.org/abs/2504.07448).
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Technical Specifications [optional]
+## Technical Specifications
 
 ### Model Architecture and Objective
 
-[More Information Needed]
+LoRI introduces a novel architecture within the LoRA framework. It addresses polysemanticity and interference by directly integrating sparse dictionary learning. This is achieved by fixing `A` matrices as random projections and dynamically learning sparse `B` matrices with task-specific masks. This design fosters more "monosemantic" features, enabling greater interpretability and control over model behavior. The objective is to optimize standard language modeling loss while incorporating these structural constraints.
 
 ### Compute Infrastructure
 
-[More Information Needed]
-
 #### Hardware
 
-[More Information Needed]
+Training was performed in a multi-GPU environment, leveraging PyTorch's Fully Sharded Data Parallel (FSDP).
 
 #### Software
 
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
+The project builds on the codebase of [dpo-rlaif](https://github.com/architsharma97/dpo-rlaif) and incorporates code from [lottery-ticket-adaptation](https://github.com/kiddyboots216/lottery-ticket-adaptation). Code generation performance on HumanEval was evaluated using the [bigcode-evaluation-harness](https://github.com/bigcode-project/bigcode-evaluation-harness).
 
-## Model Card Authors [optional]
+## Citation
 
-[More Information Needed]
+If you use LoRI in your work, please cite:
 
-## Model Card Contact
+```bibtex
+@article{zhang2025lori,
+  title={LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation},
+  author={Zhang, Juzheng and You, Jiacheng and Panda, Ashwinee and Goldstein, Tom},
+  journal={arXiv preprint arXiv:2504.07448},
+  year={2025}
+}
+```
 
-[More Information Needed]
 ### Framework versions
 
 - PEFT 0.12.0