README.md

---
datasets:
- PAPOGalaxy/PAPO_train
license: mit
pipeline_tag: image-text-to-text
library_name: transformers
---

# PAPO: Perception-Aware Policy Optimization for Multimodal Reasoning

This is the official model released for our paper [Perception-Aware Policy Optimization for Multimodal Reasoning](https://huggingface.co/papers/2507.06448).

**Project Page:** [https://mikewangwzhl.github.io/PAPO/](https://mikewangwzhl.github.io/PAPO/)
**Code:** [https://github.com/mikewangwzhl/PAPO](https://github.com/mikewangwzhl/PAPO)

## Model Version
PAPO (γ=0.01)

## Usage

You can use this model with the Hugging Face `transformers` library.

```python
from transformers import AutoProcessor, AutoModelForCausalLM
from PIL import Image
import requests

# Replace "PAPOGalaxy/PAPO" with the actual model ID if different
# For example, if it's PAPOGalaxy/PAPO-7B or PAPOGalaxy/PAPO-3B
model_id = "PAPOGalaxy/PAPO" 

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

# Example image (replace with your own image path or URL)
image_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/bee.JPG"
image = Image.open(requests.get(image_url, stream=True).raw)

# Example prompt
prompt = "What is in the image?"

# Prepare inputs following the model's chat template
messages = [
    {"role": "user", "content": [
        {"type": "image", "image": image}, 
        {"type": "text", "text": prompt}
    ]}
]
text = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=text, images=image, return_tensors="pt").to(model.device)

# Generate response
generated_ids = model.generate(**inputs, max_new_tokens=100)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(generated_text)
```