This model is part of LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation.
LoRI (LoRA with Reduced Interference) is a simple yet effective variant of LoRA for fine-tuning Large Language Models (LLMs). It addresses the challenges of parameter overhead and cross-task interference in multi-task scenarios by freezing the projection matrices A as random projections and sparsifying the matrices B using task-specific masks. This design substantially reduces the number of trainable parameters while maintaining strong task performance. Moreover, LoRI minimizes cross-task interference in adapter merging by leveraging the orthogonality between adapter subspaces, and supports continual learning by using sparsity to mitigate catastrophic forgetting. Extensive experiments across natural language understanding, mathematical reasoning, code generation, and safety alignment tasks demonstrate that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than LoRA.
LoRI (LoRA with Reduced Interference) is a parameter-efficient fine-tuning (PEFT) method designed to reduce cross-task interference in multi-task low-rank adaptation for LLMs. It achieves this by freezing projection matrices A as random projections and sparsifying matrices B with task-specific masks, significantly reducing trainable parameters (up to 95% fewer than LoRA). LoRI maintains strong task performance, minimizes interference in adapter merging through orthogonal adapter subspaces, and supports continual learning by mitigating catastrophic forgetting via sparsity.
This specific model, LoRI-D_safety_llama3_rank_32, is a LoRI-D adapter for the meta-llama/Meta-Llama-3-8B base model, specifically fine-tuned for safety alignment tasks.
meta-llama/Meta-Llama-3-8BThis adapter can be loaded with a compatible base model (meta-llama/Meta-Llama-3-8B) using the peft library to enable specialized performance on safety alignment tasks. It is intended for use in scenarios where efficient fine-tuning and reduced cross-task interference are critical.
LoRI enables effective adapter merging, allowing multiple task-specific adapters to be combined with minimal interference. It also supports continual learning, where new tasks can be learned without catastrophically forgetting previously acquired knowledge.
This model is not intended for generating harmful, biased, or unethical content. Users should exercise caution and implement appropriate safeguards when deploying the model in sensitive applications.
While LoRI aims to reduce cross-task interference, like all language models, it may still exhibit biases present in its base model and training data. Since this model is based on Llama 3, it inherits any biases or limitations present in the original Llama 3 base model. As a parameter-efficient fine-tuning method, while it aims to reduce interference, complex multi-task interactions might still present unforeseen challenges. Performance might vary on highly out-of-distribution data or tasks not well-represented in the training process.
Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. It is recommended to perform thorough testing and validation on specific use cases, especially for safety-critical applications. Further research and fine-tuning may be necessary to mitigate any identified biases or risks.
To load and use this LoRI adapter with the Meta-Llama-3-8B base model:
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
# Load the base model
base_model_name = "meta-llama/Meta-Llama-3-8B"
base_model = AutoModelForCausalLM.from_pretrained(
base_model_name,
torch_dtype=torch.bfloat16, # or torch.float16, depending on your hardware
low_cpu_mem_usage=True,
device_map="auto"
)
# Load the LoRI adapter on top of the base model
# This model is tomg-group-umd/LoRI-D_safety_llama3_rank_32
lori_adapter_name = "tomg-group-umd/LoRI-D_safety_llama3_rank_32"
model = PeftModel.from_pretrained(base_model, lori_adapter_name)
# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
# Example usage (chat completion)
messages = [
{"role": "system", "content": "You are a helpful AI assistant."},
{"role": "user", "content": "Explain the concept of quantum entanglement."},
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
outputs = model.generate(input_ids, max_new_tokens=256, temperature=0.7)
response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
print(response)
LoRI models are trained on various datasets corresponding to specific tasks. For safety alignment (which this adapter is fine-tuned for), the saferpaca dataset is used, as indicated by the saferpaca_*.sh scripts in the GitHub repository. The paper further mentions extensive experiments across Natural Language Understanding, Mathematical Reasoning, Code Generation, and Safety Alignment tasks. More details about the datasets can be found in the LoRI GitHub repository.
LoRI is implemented using Fully Sharded Data Parallel (FSDP) and can be executed in a multi-GPU environment. The training process involves two stages:
A are frozen as random projections, and matrices B are learned. This stage also extracts sparse masks.B (e.g., 90% sparsity).Training scripts for specific tasks (e.g., scripts/saferpaca_llama3_r_32.sh) are available in the GitHub repository.
bf16 or fp16).LoRI-S models.The paper presents extensive evaluations across multiple benchmarks for natural language understanding, mathematical reasoning, code generation, and safety alignment tasks.
The paper demonstrates that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than standard LoRA. In multi-task experiments, LoRI enables effective adapter merging and continual learning with reduced cross-task interference. For detailed performance metrics on safety alignment, please refer to the official paper.
LoRI introduces a novel approach to low-rank adaptation. It freezes the projection matrices A as random projections and sparsifies the matrices B using task-specific masks. This design allows for a significant reduction in the number of trainable parameters while maintaining strong task performance and minimizing cross-task interference.
The training process leverages FSDP for multi-GPU environments.
Training has been demonstrated on multi-GPU setups. Specific hardware configurations used for original training are detailed in the paper or supplementary materials.
If you use LoRI in your work, please cite:
@article{zhang2025lori,
title={LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation},
author={Zhang, Juzheng and You, Jiacheng and Panda, Ashwinee and Goldstein, Tom},
journal={arXiv preprint arXiv:2504.07448},
year={2025}
}
Base model
meta-llama/Meta-Llama-3-8B