Learning User Preferences for Image Generation Models

Website | Paper | GitHub | BibTeX

Installation

For install instructions, please see https://github.com/Mowenyii/learn-user-pref.

Usage

This model can be loaded from Hugging Face Hub as follows:

from transformers import AutoProcessor, BitsAndBytesConfig, AutoModelForVision2Seq
from peft import PeftModel
model = AutoModelForVision2Seq.from_pretrained("HuggingFaceM4/idefics2-8b")
model = PeftModel.from_pretrained(model, "wenyii/learn-user-pref")
prompt_pool=torch.load("wenyii/learn-user-pref/prompt_pool.pt") 
model.base_model.model.model.prompt_pool=prompt_pool

Please see https://github.com/Mowenyii/learn-user-pref for more detailed instructions.

Citation

If you find this repository helpful, please consider citing our work:

@article{mo2025learning,
  title={Learning User Preferences for Image Generation Model},
  author={Mo, Wenyi and Ba, Ying and Zhang, Tianyu and Bai, Yalong and Li, Biye},
  journal={arXiv preprint arXiv:2508.08220},
  year={2025}
}