task_categories:text-classification task_categories:feature-extraction task_categories:token-classification task_categories:zero-shot-classification task_categories:sentence-similarity task_categories:text-to-speech license:cc size_categories:10M<n<100M modality:text arxiv:2412.03465 doi:10.57967/hf/3602 region:us text summarization
---
license: cc
tags:
- text
- summarization
task_categories:
- text-classification
- feature-extraction
- token-classification
- zero-shot-classification
- sentence-similarity
- text-to-speech
size_categories:
- 10M<n<100M
---
# <span style="color:Red">A larger version of YT-100K dataset -> YT-30M dataset with 30 million YouTube multilingual multicategory comments is also available which can be obtained by directly emailing the author of this dataset.</span>
# Introduction

This work introduces two large-scale multilingual comment datasets, YT-30M (and YT-100K) from YouTube. The code and both the datasets: YT-30M (full) and YT-100K (randomly selected 100K sample from YT-30M) are publicly released for further research. YT-30M (YT-100K) contains 32M (100K) comments posted by YouTube channel belonging to YouTube categories. Each comment is associated with a video ID, comment ID, commenter name, commenter channel ID, comment text, upvotes, original channel ID and category of the YouTube channel (e.g., News & Politics, Science & Technology, etc.).

# Data Description

Each entry in the dataset is related to one comment for a specific YouTube video in the related category with the following columns: videoID, commentID, commenterName, commenterChannelID, comment, votes, originalChannelID, category. Each field is explained below:
```
videoID: represents the video ID in YouTube.
commentID: represents the comment ID.
commenterName: represents the name of the commenter.
commenterChannelID: represents the ID of the commenter.
comment: represents the comment text.
votes: represents the upvotes received by that comment.
originalChannelID: represents the original channel ID who posted the video.
category: represents the category of the YouTube video.
```
# Data Anonymization

The data is anonymized by removing all Personally Identifiable Information (PII). 

# Data sample
```
{
"videoID": "ab9fe84e2b2406efba4c23385ef9312a",
"commentID": "488b24557cf81ed56e75bab6cbf76fa9",
"commenterName": "b654822a96eae771cbac945e49e43cbd",
"commenterChannelID": "2f1364f249626b3ca514966e3ef3aead",
"comment": "ich fand den Handelwecker am besten",
"votes": 2,
"originalChannelID": "oc_2f1364f249626b3ca514966e3ef3aead",
"category": "entertainment"
}
```
# Multilingual data

| **Language** | **Text**                                          |

|--------------|---------------------------------------------------|

| English      | You girls are so awesome!!                        |

| Russian      | Точно так же Я стрелец                            |

| Hindi        | आज भी भाई कʏ आवाज में वही पुरानी बात है....           |

| Chinese      | 無論如何,你已經是台灣YT訂閱數之首                 |

| Bengali      | খুিন হািসনােক ভারেতর àধানমন্...                     |

| Spanish      | jajajaj esto tiene que ser una brom               |

| Portuguese   | nossa senhora!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!... |

| Malayalam    | നമസ്കാരം                                           |

| Telegu       | నమసాక్రం                                            |

| Japanese     | こんにちは                                        |

# License
[CC] (https://choosealicense.com/licenses/cc-by-4.0/#)

# Bibtex
```
@misc{dutta2024yt30mmultilingualmulticategorydataset,
      title={YT-30M: A multi-lingual multi-category dataset of YouTube comments}, 
      author={Hridoy Sankar Dutta},
      year={2024},
      eprint={2412.03465},
      archivePrefix={arXiv},
      primaryClass={cs.SI},
      url={https://arxiv.org/abs/2412.03465}, 
}
```