size_categories:10K<n<100K format:parquet modality:image modality:text library:datasets library:dask library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: id
    dtype: string
  - name: image
    dtype: image
  - name: question
    dtype: string
  - name: output
    dtype: string
  - name: ground_truth
    dtype: string
  splits:
  - name: train
    num_bytes: 4410624897.62
    num_examples: 58468
  download_size: 3936047345
  dataset_size: 4410624897.62
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
---
<img src="https://lh3.googleusercontent.com/pw/AP1GczOZcqfVlQgLnp246CO1Lfv84ezhM2DXW6_p1UYaymdXdzpFfHjnV3OR-8GZgelNXJfOjSuYe2xI-SeZv1czEXPjluV75gotsJGsFTi4n-XyQO6Ka20=w665-h666-no" alt="Italian Trulli">

# Example1

<img src="https://lh3.googleusercontent.com/pw/AP1GczNXIRoSBttb0JTYt1Dpuw15OQOQYgkITB8pDqmhVp_1QllVDUzo69cGAw_sJwrmil6azEDzcpDiRnGyF_kI00I4Squ-03wb5GJIlF-HkW035z_L0CU=w769-h356-no" alt="Italian Trulli">


## Input:

Please answer the question below, explaining your reasoning step by step before providing the final answer.

Question: Are there enough straws for every cup ?

A. yes

B. no

## Output:


`<SUMMARY/>`The question asks whether there are enough straws to provide one for each cup depicted in an image.
To answer, we <span style="color:blue">need to count</span> the number of straws and cups separately and then <span style="color:blue">compare those quantities</span>.`</SUMMARY>`

`<CAPTION>`The image shows three red-and-white striped straws and five blue-and-white striped cups filled with a brown liquid.
The cups are arranged in a horizontal row, and the straws are arranged vertically above them.`</CAPTION>`

`<REASONING>`1. **Identify the objects:** The image contains straws and cups. We need to determine the number of each.

2. **Count the straws:** Carefully examine the image. There are <span style="color:blue">three straws</span> in total. Each straw is distinct and easily identifiable.

3. **Count the cups:** Now count the number of cups. There are <span style="color:blue">five cups</span>, each filled with a dark-brown liquid and featuring a similar striped design.

4. **Compare quantities:** We compare the number of straws (3) to the number of cups (5). <span style="color:blue">The number of cups (5) is greater than the number of straws (3)</span>.

5. **Draw a conclusion:** Since there are fewer straws than cups, there <span style="color:blue">aren't enough</span> straws for every cup to have one.`</REASONING>`

`<CONCLUSION>`B. no`</CONCLUSION>`

## Llava-o1's creator just gave us a like. Thank you for your interesting contributions to the community 😊!

<img src="https://lh3.googleusercontent.com/pw/AP1GczNFuVrsyxgu9rQNRUJP0xjZDJQv_tZJfDyrkob6NBiGeSBTqwxG_yRaMLPWpK_P-kqurYhHjRlms5TZXtFQO0aQfi3BmVnRp53S3loreQ4kCuGhthw=w769-h272-no" alt="Italian Trulli" width=400>

## References

G. Xu, P. Jin, H. Li, Y. Song, L. Sun, and L. Yuan, ‘LLaVA-CoT: Let Vision Language Models Reason Step-by-Step’, arXiv [cs.CV]. 2024.

W. Wang et al., ‘Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization’, arXiv [cs.CL]. 2024.

**We are continuing to refine reasoning errors in this dataset using Gemini Pro to create a complete DPO dataset. Stay tuned!**

