size_categories:1K<n<10K format:parquet modality:image modality:text library:datasets library:dask library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: image
    dtype: image
  - name: question
    dtype: string
  - name: answers
    sequence: string
  - name: image_classes
    sequence: string
  splits:
  - name: test
    num_bytes: 1437028491.0
    num_examples: 5000
  download_size: 919339170
  dataset_size: 1437028491.0
configs:
- config_name: default
  data_files:
  - split: test
    path: data/test-*
---

Evaluate with [VQA Accuracy](https://visualqa.org/evaluation.html): Note! A question might have various correct answers! So if more than 3 answers are the same, then scores 1; otherwise, scores with proportion.

(here is a evaluation function that you can use)

```python
def vqa_accuracy(predictions, ground_truths_list):
    accuracies = []
    for pred, ground_truths in zip(predictions, ground_truths_list):
        answer_count = collections.Counter(ground_truths)
        accuracy = min(1.0, answer_count[pred] / 3.0)
        accuracies.append(accuracy)
    return sum(accuracies) / len(accuracies)
```
