File size: 1,887 Bytes
35a9e8b 5415fff 35a9e8b 5415fff 35a9e8b 5415fff 35a9e8b 5415fff 35a9e8b 5415fff |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
---
datasets:
- PAPOGalaxy/PAPO_train
license: mit
pipeline_tag: image-text-to-text
library_name: transformers
---
# PAPO: Perception-Aware Policy Optimization for Multimodal Reasoning
This is the official model released for our paper [Perception-Aware Policy Optimization for Multimodal Reasoning](https://huggingface.co/papers/2507.06448).
**Project Page:** [https://mikewangwzhl.github.io/PAPO/](https://mikewangwzhl.github.io/PAPO/)
**Code:** [https://github.com/mikewangwzhl/PAPO](https://github.com/mikewangwzhl/PAPO)
## Model Version
PAPO (γ=0.01)
## Usage
You can use this model with the Hugging Face `transformers` library.
```python
from transformers import AutoProcessor, AutoModelForCausalLM
from PIL import Image
import requests
# Replace "PAPOGalaxy/PAPO" with the actual model ID if different
# For example, if it's PAPOGalaxy/PAPO-7B or PAPOGalaxy/PAPO-3B
model_id = "PAPOGalaxy/PAPO"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
# Example image (replace with your own image path or URL)
image_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/bee.JPG"
image = Image.open(requests.get(image_url, stream=True).raw)
# Example prompt
prompt = "What is in the image?"
# Prepare inputs following the model's chat template
messages = [
{"role": "user", "content": [
{"type": "image", "image": image},
{"type": "text", "text": prompt}
]}
]
text = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=text, images=image, return_tensors="pt").to(model.device)
# Generate response
generated_ids = model.generate(**inputs, max_new_tokens=100)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(generated_text)
``` |