language:en license:mit size_categories:10M<n<100M region:us
---
license: mit
language:
- en
size_categories:
- 10M<n<100M
---

# GDSC 2024
This dataset contains the results from the Capgemini Global Data Science Challenge (GDSC) 2024 Arena Battles, where AI education policy experts competed to provide the best answers to questions about global education trends and literacy.  
**Quick Links:**
- [Case study](https://bit.ly/4gueMda)
- [GDSC Overview](https://www.capgemini.com/insights/research-library/global-data-science-challenge/)
- [GDSC 7 Overview Video (Short)](https://www.youtube.com/watch?v=R-HqqEoUfVo)
- [GDSC 7 Overview Video (Long)](https://www.youtube.com/watch?v=Ra0--tYSt4s)
- [GDSC Website](https://gdsc.ce.capgemini.com/)

# Background
The Capgemini Global Data Science Challenge (GDSC) is an annual, purpose-driven hackathon that brings
together thousands of participants globally to tackle real-world challenges using AI. In 2024, GDSC partnered
with UNESCO and AWS to address the global learning crisis and advance the [UN's Sustainable Development Goal 4](https://sdgs.un.org/goals/goal4):
ensuring inclusive and equitable quality education.

This dataset contains the results of arena battles conducted between October and November 2024, where
different AI models competed to provide the best answers to education-related questions based on the
PIRLS 2021 dataset.

# Dataset Contents
The dataset consists of four tables:

1. Questions
  - Columns: `question_id`, `question`  
  - Contains questions curated by UNESCO, GDSC organizers, and participants' proposals.

2. Answers
  - Columns: `answer_id`, `answer`  
  - Contains answers generated by participants' solutions.

3. Battles
  - Columns: `evaluator_id`, `left_answer_id`, `right_answer_id`, `battle_outcome`, `timestamp`
  - Records battle outcomes between different AI solutions
  - Evaluator ID is anonymized
  - Special cases:
    - auto-evaluated: battles where at least one solution failed to provide an answer within the time limit.
    - "anonymous: evaluations by non-logged-in users.

4. Questions and Answers
  - Columns: `question_id`, `answer_id`, `submission_id`
  - Join table linking answers to their respective questions.

# How Arena Battles Work
The arena is a competitive evaluation platform where different AI solutions (models) compete against each other to 
provide the best answers to education-related questions. Here's how it works:

1. Battle Setup
  - Two AI models are randomly selected based on their current ratings
  - Both models receive the same question
  - Each model generates an answer within a time limit
  - The answers are presented side-by-side to evaluators

2. Battle Resolution
  - If both models provide answers on time:
    - Human evaluators compare both answers
    - Evaluators choose which answer they prefer or mark it as a draw
    - The outcome affects both models' ratings
  - If one model fails to respond in time:
    - It automatically loses the battle
  - If both models fail to respond:
    - The battle is marked as a draw

3. Rating System
  - Models are ranked using the [Glicko-2](https://de.wikipedia.org/wiki/Glicko-System) rating system
  - Winning battles increases a model's rating
  - Losing battles decreases a model's rating
  - The amount of rating change depends on:
    - The rating difference between the models
    - The number of battles each model has participated in
    - The time since their last battle

# Evaluation Process
Battles were evaluated through two methods:
1. Human Evaluation: IT consultants from Capgemini and education experts from UNESCO evaluated answers based on:
  - Clarity of response
  - Correctness and consistency of information
  - Overall quality of the answer

2. Automatic Evaluation:
  - If one solution failed to respond within the time limit, it automatically lost
  - If both solutions timed out, the battle was marked as a draw
  - The scoring system uses the [Glicko-2](https://de.wikipedia.org/wiki/Glicko-System) rating algorithm to rank solutions based on their battle performances.

---

# Example questions and answers
### Q: What percentage of parents reported that their child's learning progress was adversely affected ""a lot"" by the pandemic in the PIRLS 2021 study?
### A: Pandemic Impact on Children's Learning Progress in PIRLS 2021

According to the PIRLS 2021 study, **26.43%** of parents reported that their child's learning progress was adversely affected "a lot" by the pandemic.

This percentage is based on the following data from the study:

- Total number of valid parent responses: 235,688
- Number of parents who reported 'a lot' of adverse effects: 62,282

To calculate this percentage:
(62,282 / 235,688) * 100 = 26.43%

This means that approximately one out of every four parents who participated in the PIRLS 2021 study felt that the COVID-19 pandemic had a significant negative impact on their child's learning progress.

It's important to note that this data represents the parents'

---

### Q: Show a plot of the correlation of a countries GDP and its readings skills according to the PIRLS 2021 study
### A: Analysis of Reading Skills and GDP Correlation

Thank you for your interesting question about the correlation between countries' GDP and their reading skills according to the PIRLS 2021 study. As a data analyst, I'm excited to dive into this topic, but I need to clarify a few points and provide some context.

## Current Data Limitations

Unfortunately, I don't have direct access to GDP data in conjunction with the PIRLS 2021 study results. The query I ran only provided reading scores for participating countries and regions. To create the plot you're asking for, we would need to combine two datasets:

1. PIRLS 2021 reading scores
2. GDP data for the same countries in the same year

## What We Know

Based on the PIRLS 2021 data we have:

- 65 countries/regions participated in the study
- Reading scores range from 267.19 (South Africa) to 599.01 (Moscow City, Russian Federation)
- Generally, more developed regions tend to have higher scores

## Next Steps for Analysis

To properly answer your question and create the plot, we would need to:

1. Obtain GDP data for the countries in the PIRLS 2021 study
2. Match the GDP data with our existing reading scores
3. Use a statistical tool to create a scatter plot
4. Calculate the correlation coefficient between GDP and reading scores

## Preliminary Insights

While we can't draw definitive conclusions without the complete dataset, based on general trends in educational outcomes and economic development, we might expect to see a positive correlation between GDP and reading skills. However, it's important to note that this relationship may not be strictly linear and could be influenced by various factors such as:

- Education system efficiency
- Cultural emphasis on reading
- Income inequality within countries
- Investment in education as a percentage of GDP

## Follow-up Question

To delve deeper into this analysis, you might want to consider the following:

Do you know how to obtain GDP data for the countries in the PIRLS 2021 study? Ask me a following question: 'What reliable sources can we use to gather GDP data for the countries in the PIRLS 2021 study, and how can we ensure the data is for the same year as the study?'

By answering this question, we can take the next step in creating the correlation plot you're interested in and provide a more comprehensive analysis of the relationship between GDP and reading skills.


---


# License
This dataset is released under the MIT License. See the LICENSE file for details.

# Citation
If you use this dataset in your research, please cite:
```
Copy@dataset{gdsc2024,  title={GDSC 2024 Arena Battles Dataset},  author={Capgemini Global Data Science Challenge Team},  year={2024},  url={https://bit.ly/4gueMda}}
```
