Improve model card for LoRI-S_safety_llama3_rank_64

README.md

@@ -2,192 +2,175 @@
 base_model: meta-llama/Meta-Llama-3-8B
 library_name: peft
 pipeline_tag: text-generation
+license: apache-2.0
 ---
 
 # Model Card for LoRI-S_safety_llama3_rank_64
 
 This model is part of [LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation](https://arxiv.org/abs/2504.07448).
 
-<!-- Provide a quick summary of what the model is/does. -->
-
+**LoRI** (LoRA with Reduced Interference) is a simple yet effective variant of Low-Rank Adaptation (LoRA) for fine-tuning Large Language Models (LLMs). It addresses the challenges of overhead and parameter interference in multi-task scenarios by freezing the projection matrices `A` as random projections and sparsifying the matrices `B` using task-specific masks. This design substantially reduces the number of trainable parameters (up to 95% fewer than LoRA) while maintaining strong task performance and minimizing cross-task interference. LoRI also supports continual learning by using sparsity to mitigate catastrophic forgetting.
 
+<div align="center">
+    <img src="https://github.com/juzhengz/LoRI/raw/main/LoRI.png" alt="LoRI" width="80%">
+</div>
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
-
-
+LoRI-S_safety_llama3_rank_64 is a LoRI-S (Sparse) adapter specifically fine-tuned for safety alignment tasks. It is built upon the `meta-llama/Meta-Llama-3-8B` base model and utilizes a low-rank adapter with `r=64`. This model is part of the LoRI family of models presented in the paper cited above, demonstrating improved efficiency and reduced cross-task interference in multi-task and continual learning settings.
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+-   **Developed by:** Juzheng Zhang, Jiacheng You, Ashwinee Panda, Tom Goldstein
+-   **Model type:** Low-Rank Adaptation (LoRA) adapter for Causal Language Models, specifically LoRI-S (Sparse)
+-   **Language(s) (NLP):** English
+-   **License:** Apache 2.0
+-   **Finetuned from model:** `meta-llama/Meta-Llama-3-8B`
 
-### Model Sources [optional]
+### Model Sources
 
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+-   **Repository:** [https://github.com/juzhengz/LoRI/](https://github.com/juzhengz/LoRI/)
+-   **Paper:** [https://arxiv.org/abs/2504.07448](https://arxiv.org/abs/2504.07448)
+-   **Hugging Face Collection:** [https://huggingface.co/collections/tomg-group-umd/lori-adapters-67f795549d792613e1290011](https://huggingface.co/collections/tomg-group-umd/lori-adapters-67f795549d792613e1290011)
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
+This model is intended to be used as a PEFT adapter with the `meta-llama/Meta-Llama-3-8B` base model for text generation tasks, particularly for improving safety alignment capabilities. It can be loaded with the `peft` library and combined with the base LLM for efficient inference.
 
-### Downstream Use [optional]
+### Downstream Use
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
+LoRI enables effective adapter merging, allowing multiple task-specific adapters to be combined into a single model while minimizing performance degradation due to cross-task interference. It also supports continual learning scenarios, where the model needs to adapt to new tasks sequentially without suffering from catastrophic forgetting of previously learned tasks.
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+This model is not intended for use as a standalone LLM; it requires the `meta-llama/Meta-Llama-3-8B` base model to function. As a fine-tuned model, it may exhibit biases present in its training data or the base model. Users should exercise caution and perform additional evaluations for critical or sensitive applications, and not rely on it for factually accurate or harmless outputs without further safeguards.
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
+While fine-tuned for safety alignment, this LoRI adapter, like all large language models, may still generate harmful, biased, or nonsensical content. Its performance and robustness are highly dependent on the quality and diversity of the safety alignment data it was trained on. Potential risks include:
+-   Generation of unintended or inappropriate responses despite safety training.
+-   Reinforcement of societal biases present in the base model or the training data.
+-   Limitations in generalizing to novel or highly out-of-distribution safety scenarios.
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+Users should always:
+-   Carefully evaluate the model's outputs in their specific use cases.
+-   Implement additional safety filters and human oversight for sensitive applications.
+-   Be aware that the model's behavior is influenced by both the base model and the LoRI adapter's training.
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
-[More Information Needed]
+Install the necessary libraries:
+```bash
+pip install transformers peft
+```
+
+Use the code below to load and use the LoRI adapter:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+import torch
+
+# Load the base model (Meta-Llama-3-8B)
+# Ensure you have access to Llama-3 models on Hugging Face
+base_model_name = "meta-llama/Meta-Llama-3-8B"
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_name,
+    torch_dtype=torch.bfloat16, # Use bfloat16 for better performance if supported
+    low_cpu_mem_usage=True,
+    device_map="auto"
+)
+
+# Load the LoRI adapter
+adapter_name = "tomg-group-umd/LoRI-S_safety_llama3_rank_64"
+model = PeftModel.from_pretrained(base_model, adapter_name)
+
+# Load the tokenizer
+tokenizer = AutoTokenizer.from_pretrained(base_model_name)
+
+# Example usage for text generation
+prompt = "Explain why it is important for AI models to prioritize user safety."
+messages = [
+    {"role": "user", "content": prompt},
+]
+
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+outputs = model.generate(input_ids, max_new_tokens=256, temperature=0.7, do_sample=True)
+response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(response)
+```
 
 ## Training Details
 
-### Training Data
+LoRI models are trained using a two-stage process: `LoRI-D` (Dense) and `LoRI-S` (Sparse). The training is implemented using Fully Sharded Data Parallel (FSDP) and can be executed in a multi-GPU environment.
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+### Training Data
 
-[More Information Needed]
+LoRI models are trained on various datasets depending on the task. For safety alignment, this model was trained using the `saferpaca` script, which implies a safety-related dataset. More details on the specific datasets used for different tasks (Natural Language Understanding, Code Generation, Mathematical Reasoning, and Safety Alignment) can be found in the [LoRI GitHub repository](https://github.com/juzhengz/LoRI/).
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
+1.  **LoRI-D (Dense) Training:** Initial training where projection matrices `A` are frozen as random projections and `B` matrices are dense.
+2.  **Sparse Mask Extraction:** After `LoRI-D` training, sparse masks are extracted from the learned `B` matrices.
+3.  **LoRI-S (Sparse) Training:** Continual training is performed with the extracted sparse masks applied to `B` matrices to achieve a desired sparsity (e.g., 90% sparsity), while keeping `A` frozen. This model, `LoRI-S_safety_llama3_rank_64`, is the result of the `LoRI-S` training stage.
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+-   **Adapter Rank (r):** 64
+-   **LoRA Alpha:** 128
+-   **LoRA Dropout:** 0.05
+-   **Sparsity:** 90% (for LoRI-S stage)
+-   **Training regime:** Mixed precision (using FSDP for multi-GPU setups).
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+LoRI has been extensively evaluated across various tasks, including natural language understanding, mathematical reasoning, code generation, and safety alignment.
 
 ### Results
 
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
+Extensive experiments demonstrate that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than LoRA. In multi-task experiments, LoRI enables effective adapter merging and continual learning with reduced cross-task interference.
 
-## Technical Specifications [optional]
+## Technical Specifications
 
 ### Model Architecture and Objective
 
-[More Information Needed]
+LoRI introduces a novel architecture within the PEFT paradigm. It leverages random projections for matrix `A` and task-specific sparse masks for matrix `B`, integrated directly into the training process. The objective is to achieve monosemantic adaptations that reduce cross-task interference and enable efficient multi-task and continual learning.
 
 ### Compute Infrastructure
 
-[More Information Needed]
-
 #### Hardware
 
-[More Information Needed]
+LoRI can be trained and inferred efficiently in multi-GPU environments, leveraging technologies like Fully Sharded Data Parallel (FSDP).
 
 #### Software
 
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
+The implementation builds on PyTorch, Hugging Face Transformers, and PEFT libraries.
 
-[More Information Needed]
+-   **Framework versions:** PEFT 0.12.0
 
-## Glossary [optional]
+## Citation
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+If you use LoRI in your work, please cite:
 
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
+```bibtex
+@article{zhang2025lori,
+  title={LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation},
+  author={Zhang, Juzheng and You, Jiacheng and Panda, Ashwinee and Goldstein, Tom},
+  journal={arXiv preprint arXiv:2504.07448},
+  year={2025}
+}
+```
 
 ## Model Card Contact
 
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.12.0
+For questions or inquiries, please refer to the contact information on the [official LoRI GitHub repository](https://github.com/juzhengz/LoRI/). You may also contact the authors listed in the paper.