README.md

---
license: mit
base_model:
- Qwen/Qwen2.5-VL-7B-Instruct
pipeline_tag: image-text-to-text
---
# Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation
[![arXiv](https://img.shields.io/badge/arXiv-2508.12680-b31b1b?logo=arxiv&logoColor=white)](https://arxiv.org/abs/2508.12680)
[![GitHub](https://img.shields.io/badge/GitHub-Repo-181717?logo=github&logoColor=white)](https://github.com/yuh-zha/Vision-G1)
[![HuggingFace](https://img.shields.io/badge/HuggingFace-Model-FFD21E?logo=huggingface&logoColor=black)](https://huggingface.co/yzha/vision-g1)
## Introduction
We present the reasoning VLM: Vision-G1, which is trained through multi-domain data curation. Specifically, we include training data from 46 data sources across 8 dimensions. This repo includes training code, data preprocessing scripts, and evaluation scripts.
## Installation
To enable the inference of Vision-G1, you can simply install the latest transformers library:
```bash
pip install transformers
```
Optionally, if you wish to accelerate the inference, please install vllm:
```bash
pip install vllm
```
To support training, installing verl is required:
```bash
cd training/
pip install -r requirements.txt
pip install flash_attn==2.7.4.post1 --no-build-isolation
pip install -e .
```
## Inference
Our model follows the transformers format. You can load it with standard transformers APIs:
```python
from transformers import Qwen2_5_VLForConditionalGeneration, AutoTokenizer, AutoProcessor
from qwen_vl_utils import process_vision_info

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "yzha/vision-g1",
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
    device_map="auto",
)

# default processer
processor = AutoProcessor.from_pretrained("yzha/vision-g1")

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "<path to image>",
            },
            {"type": "text", "text": "<Question>"},
        ],
    }
]

# Preparation for inference
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
)
inputs = inputs.to("cuda")

# Inference: Generation of the output
generated_ids = model.generate(**inputs, max_new_tokens=128)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)

```
For faster inference, the model also supports vllm server. Simply start a vllm host by:
```bash
vllm serve yzha/vision-g1 \
  --host 0.0.0.0 --port 8000 \
  --max-model-len 8192 \
```
## Training
The training scripts are in `training/examples/grpo_trainer`. For single node training, please use `single_node_vision_g1.sh`. For distributed training, please use `distributed_vision_g1.sh`. Before launching the training, you need to specify the training and validation data path in the bash script, by setting `train_files` and `test_files`. To launch the training, 
### Single node
```bash
cd training/
bash examples/grpo_trainer/single_node_vision_g1.sh
```
### Multiple nodes
```bash
cd training/
sbatch examples/grpo_trainer/distributed_vision_g1.sh
```
## Data
TBD
## Evaluation
TBD
## Acknowledgement
We use [verl](https://github.com/volcengine/verl) as the codebase to build our training framework.
## Citation
If you are interested in our work, please cite:
```
@article{zha2025vision,
  title={Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation},
  author={Zha, Yuheng and Zhou, Kun and Wu, Yujia and Wang, Yushu and Feng, Jie and Xu, Zhi and Hao, Shibo and Liu, Zhengzhong and Xing, Eric P and Hu, Zhiting},
  journal={arXiv preprint arXiv:2508.12680},
  year={2025}
}
```