task_categories:text-classification task_categories:token-classification task_categories:question-answering task_categories:summarization task_categories:text-generation task_ids:sentiment-analysis task_ids:topic-classification task_ids:named-entity-recognition task_ids:language-modeling task_ids:text-scoring task_ids:multi-class-classification task_ids:multi-label-classification task_ids:extractive-qa task_ids:news-articles-summarization multilinguality:multilingual source_datasets:original license:mit size_categories:10K<n<100K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: mit
multilinguality:
  - multilingual
source_datasets:
  - original
task_categories:
  - text-classification
  - token-classification
  - question-answering
  - summarization
  - text-generation
task_ids:
  - sentiment-analysis
  - topic-classification
  - named-entity-recognition
  - language-modeling
  - text-scoring
  - multi-class-classification
  - multi-label-classification
  - extractive-qa
  - news-articles-summarization
---


# Bittensor Subnet 13 Reddit Dataset

<center>
    <img src="https://huggingface.co/datasets/macrocosm-os/images/resolve/main/bittensor.png" alt="Data-universe: The finest collection of social media data the web has to offer">
</center>

<center>
    <img src="https://huggingface.co/datasets/macrocosm-os/images/resolve/main/macrocosmos-black.png" alt="Data-universe: The finest collection of social media data the web has to offer">
</center>


## Dataset Description

- **Repository:** arrmlet/reddit_dataset_58
- **Subnet:** Bittensor Subnet 13
- **Miner Hotkey:** vlad_hotkey

### Dataset Summary

This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed Reddit data. The data is continuously updated by network miners, providing a real-time stream of Reddit content for various analytical and machine learning tasks.
For more information about the dataset, please visit the [official repository](https://github.com/macrocosm-os/data-universe).

### Supported Tasks

The versatility of this dataset allows researchers and data scientists to explore various aspects of social media dynamics and develop innovative applications. Users are encouraged to leverage this data creatively for their specific research or business needs.
For example:

- Sentiment Analysis
- Topic Modeling
- Community Analysis
- Content Categorization

### Languages

Primary language: Datasets are mostly English, but can be multilingual due to decentralized ways of creation.

## Dataset Structure

### Data Instances

Each instance represents a single Reddit post or comment with the following fields:


### Data Fields

- `text` (string): The main content of the Reddit post or comment.
- `label` (string): Sentiment or topic category of the content.
- `dataType` (string): Indicates whether the entry is a post or a comment.
- `communityName` (string): The name of the subreddit where the content was posted.
- `datetime` (string): The date when the content was posted or commented.
- `username_encoded` (string): An encoded version of the username to maintain user privacy.
- `url_encoded` (string): An encoded version of any URLs included in the content.

### Data Splits

This dataset is continuously updated and does not have fixed splits. Users should create their own splits based on their requirements and the data's timestamp.

## Dataset Creation

### Source Data

Data is collected from public posts and comments on Reddit, adhering to the platform's terms of service and API usage guidelines.

### Personal and Sensitive Information

All usernames and URLs are encoded to protect user privacy. The dataset does not intentionally include personal or sensitive information.

## Considerations for Using the Data

### Social Impact and Biases

Users should be aware of potential biases inherent in Reddit data, including demographic and content biases. This dataset reflects the content and opinions expressed on Reddit and should not be considered a representative sample of the general population.

### Limitations

- Data quality may vary due to the nature of media sources.
- The dataset may contain noise, spam, or irrelevant content typical of social media platforms.
- Temporal biases may exist due to real-time collection methods.
- The dataset is limited to public subreddits and does not include private or restricted communities.

## Additional Information

### Licensing Information

The dataset is released under the MIT license. The use of this dataset is also subject to Reddit Terms of Use.

### Citation Information

If you use this dataset in your research, please cite it as follows:

```
@misc{arrmlet2024datauniversereddit_dataset_58,
        title={The Data Universe Datasets: The finest collection of social media data the web has to offer},
        author={arrmlet},
        year={2024},
        url={https://huggingface.co/datasets/arrmlet/reddit_dataset_58},
        }
```

### Contributions

To report issues or contribute to the dataset, please contact the miner or use the Bittensor Subnet 13 governance mechanisms.

## Dataset Statistics

[This section is automatically updated]

- **Total Instances:** 28506
- **Date Range:** 2019-07-24T00:00:00Z to 2024-10-23T00:00:00Z
- **Last Updated:** 2024-10-24T13:24:08Z

### Data Distribution

- Posts: 13.68%
- Comments: 86.32%

### Top 10 Subreddits

For full statistics, please refer to the `stats.json` file in the repository.

| Rank | Topic | Total Count | Average Percentage |
|------|-------|-------------|--------------------|
| 1 | r/wallstreetbets | 6996 | 24.54% |
| 2 | r/Bitcoin | 4394 | 15.41% |
| 3 | r/ethtrader | 4375 | 15.35% |
| 4 | r/CryptoCurrency | 3994 | 14.01% |
| 5 | r/solana | 2257 | 7.92% |
| 6 | r/CryptoMarkets | 1437 | 5.04% |
| 7 | r/btc | 1167 | 4.09% |
| 8 | r/Monero | 1027 | 3.60% |
| 9 | r/Polkadot | 972 | 3.41% |
| 10 | r/bittensor_ | 601 | 2.11% |


## Update History

| Date | New Instances | Total Instances |
|------|---------------|-----------------|
| 2024-10-24T13:24:08Z | 28506 | 28506 |
