Improve model card for LoRI-D_safety_llama3_rank_32

README.md

@@ -2,192 +2,165 @@
 base_model: meta-llama/Meta-Llama-3-8B
 library_name: peft
 pipeline_tag: text-generation
+license: apache-2.0
+language: en
+tags:
+- peft
+- lora
+- safety-llm
+- llm
 ---
 
 # Model Card for LoRI-D_safety_llama3_rank_32
 
 This model is part of [LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation](https://arxiv.org/abs/2504.07448).
 
-<!-- Provide a quick summary of what the model is/does. -->
-
+**LoRI (LoRA with Reduced Interference)** is a simple yet effective variant of LoRA for fine-tuning Large Language Models (LLMs). It addresses the challenges of parameter overhead and cross-task interference in multi-task scenarios by freezing the projection matrices `A` as random projections and sparsifying the matrices `B` using task-specific masks. This design substantially reduces the number of trainable parameters while maintaining strong task performance. Moreover, LoRI minimizes cross-task interference in adapter merging by leveraging the orthogonality between adapter subspaces, and supports continual learning by using sparsity to mitigate catastrophic forgetting. Extensive experiments across natural language understanding, mathematical reasoning, code generation, and safety alignment tasks demonstrate that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than LoRA.
 
+<div align="center">
+    <img src="https://github.com/juzhengz/LoRI/raw/main/LoRI.png" alt="LoRI" width="80%">
+</div>
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
-
-
+LoRI (LoRA with Reduced Interference) is a parameter-efficient fine-tuning (PEFT) method designed to reduce cross-task interference in multi-task low-rank adaptation for LLMs. It achieves this by freezing projection matrices `A` as random projections and sparsifying matrices `B` with task-specific masks, significantly reducing trainable parameters (up to 95% fewer than LoRA). LoRI maintains strong task performance, minimizes interference in adapter merging through orthogonal adapter subspaces, and supports continual learning by mitigating catastrophic forgetting via sparsity.
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+This specific model, `LoRI-D_safety_llama3_rank_32`, is a `LoRI-D` adapter for the `meta-llama/Meta-Llama-3-8B` base model, specifically fine-tuned for safety alignment tasks.
 
-### Model Sources [optional]
+-   **Developed by:** Juzheng Zhang, Jiacheng You, Ashwinee Panda, Tom Goldstein
+-   **Model type:** PEFT LoRA adapter for causal language models (LLMs)
+-   **Language(s) (NLP):** English (primary for safety alignment task, built on a multilingual base model)
+-   **License:** Apache 2.0
+-   **Finetuned from model:** `meta-llama/Meta-Llama-3-8B`
 
-<!-- Provide the basic links for the model. -->
+### Model Sources
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+-   **Repository:** [https://github.com/juzhengz/LoRI/](https://github.com/juzhengz/LoRI/)
+-   **Paper:** [https://arxiv.org/abs/2504.07448](https://arxiv.org/abs/2504.07448)
+-   **Hugging Face Collection:** [https://huggingface.co/collections/tomg-group-umd/lori-adapters-67f795549d792613e1290011](https://huggingface.co/collections/tomg-group-umd/lori-adapters-67f795549d792613e1290011)
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
+This adapter can be loaded with a compatible base model (`meta-llama/Meta-Llama-3-8B`) using the `peft` library to enable specialized performance on safety alignment tasks. It is intended for use in scenarios where efficient fine-tuning and reduced cross-task interference are critical.
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+### Downstream Use
 
-[More Information Needed]
+LoRI enables effective adapter merging, allowing multiple task-specific adapters to be combined with minimal interference. It also supports continual learning, where new tasks can be learned without catastrophically forgetting previously acquired knowledge.
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+This model is not intended for generating harmful, biased, or unethical content. Users should exercise caution and implement appropriate safeguards when deploying the model in sensitive applications.
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
+While LoRI aims to reduce cross-task interference, like all language models, it may still exhibit biases present in its base model and training data. Since this model is based on Llama 3, it inherits any biases or limitations present in the original Llama 3 base model. As a parameter-efficient fine-tuning method, while it aims to reduce interference, complex multi-task interactions might still present unforeseen challenges. Performance might vary on highly out-of-distribution data or tasks not well-represented in the training process.
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. It is recommended to perform thorough testing and validation on specific use cases, especially for safety-critical applications. Further research and fine-tuning may be necessary to mitigate any identified biases or risks.
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
-[More Information Needed]
+To load and use this LoRI adapter with the `Meta-Llama-3-8B` base model:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+import torch
+
+# Load the base model
+base_model_name = "meta-llama/Meta-Llama-3-8B"
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_name,
+    torch_dtype=torch.bfloat16, # or torch.float16, depending on your hardware
+    low_cpu_mem_usage=True,
+    device_map="auto"
+)
+
+# Load the LoRI adapter on top of the base model
+# This model is tomg-group-umd/LoRI-D_safety_llama3_rank_32
+lori_adapter_name = "tomg-group-umd/LoRI-D_safety_llama3_rank_32" 
+model = PeftModel.from_pretrained(base_model, lori_adapter_name)
+
+# Load the tokenizer
+tokenizer = AutoTokenizer.from_pretrained(base_model_name)
+
+# Example usage (chat completion)
+messages = [
+    {"role": "system", "content": "You are a helpful AI assistant."},
+    {"role": "user", "content": "Explain the concept of quantum entanglement."},
+]
+
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+outputs = model.generate(input_ids, max_new_tokens=256, temperature=0.7)
+response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
+print(response)
+```
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
+LoRI models are trained on various datasets corresponding to specific tasks. For safety alignment (which this adapter is fine-tuned for), the `saferpaca` dataset is used, as indicated by the `saferpaca_*.sh` scripts in the GitHub repository. The paper further mentions extensive experiments across Natural Language Understanding, Mathematical Reasoning, Code Generation, and Safety Alignment tasks. More details about the datasets can be found in the [LoRI GitHub repository](https://github.com/juzhengz/LoRI/).
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
+LoRI is implemented using Fully Sharded Data Parallel (FSDP) and can be executed in a multi-GPU environment. The training process involves two stages:
+1.  **LoRI-D (Dynamic):** Initial training where projection matrices `A` are frozen as random projections, and matrices `B` are learned. This stage also extracts sparse masks.
+2.  **LoRI-S (Sparse):** Continues training with the sparse masks applied to matrices `B` (e.g., 90% sparsity).
 
+Training scripts for specific tasks (e.g., `scripts/saferpaca_llama3_r_32.sh`) are available in the [GitHub repository](https://github.com/juzhengz/LoRI/tree/main/scripts).
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+-   **Training regime:** Mixed precision (specifics depend on environment/GPU, typically `bf16` or `fp16`).
+-   **Adapter Ranks:** 32 (for this specific model) or 64.
+-   **Sparsity:** Up to 90% for `LoRI-S` models.
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+The paper presents extensive evaluations across multiple benchmarks for natural language understanding, mathematical reasoning, code generation, and safety alignment tasks.
 
 ### Results
 
-[More Information Needed]
-
-#### Summary
-
-
+The paper demonstrates that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than standard LoRA. In multi-task experiments, LoRI enables effective adapter merging and continual learning with reduced cross-task interference. For detailed performance metrics on safety alignment, please refer to the [official paper](https://arxiv.org/abs/2504.07448).
 
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Technical Specifications [optional]
+## Technical Specifications
 
 ### Model Architecture and Objective
 
-[More Information Needed]
+LoRI introduces a novel approach to low-rank adaptation. It freezes the projection matrices `A` as random projections and sparsifies the matrices `B` using task-specific masks. This design allows for a significant reduction in the number of trainable parameters while maintaining strong task performance and minimizing cross-task interference.
 
 ### Compute Infrastructure
 
-[More Information Needed]
+The training process leverages FSDP for multi-GPU environments.
 
 #### Hardware
 
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
+Training has been demonstrated on multi-GPU setups. Specific hardware configurations used for original training are detailed in the paper or supplementary materials.
 
-## Model Card Authors [optional]
+## Citation
 
-[More Information Needed]
+If you use LoRI in your work, please cite:
 
-## Model Card Contact
+```bibtex
+@article{zhang2025lori,
+  title={LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation},
+  author={Zhang, Juzheng and You, Jiacheng and Panda, Ashwinee and Goldstein, Tom},
+  journal={arXiv preprint arXiv:2504.07448},
+  year={2025}
+}
+```
 
-[More Information Needed]
 ### Framework versions
 
 - PEFT 0.12.0