language:en size_categories:1M<n<10M format:parquet modality:tabular modality:text library:datasets library:dask library:mlcroissant library:polars arxiv:2502.10341 region:us
---
language:
- en
---
# WebOrganizer/TopicAnnotations-Llama-3.1-8B

[[Paper](https://arxiv.org/abs/2502.10341)] [[Website](https://weborganizer.allenai.org)] [[GitHub](https://github.com/CodeCreator/WebOrganizer)]

This dataset contains 1M web pages annotated with topic labels by the Llama-3.1-8B model. The web pages are a sample of the DCLM RefinedWeb reproduction. It is used as first-stage training data for the [WebOrganizer/TopicClassifier](https://huggingface.co/WebOrganizer/TopicClassifier).

## Dataset Structure

Each example contains the following fields:
- `text`: The text content of the web page
- `url`: The URL of the web page
- `top_choice_index`: Index of the most likely topic label
- `top_choice`: The most likely topic label (using the description in the [taxonomy config](https://github.com/CodeCreator/WebOrganizer/blob/main/define_domains/taxonomies/topics.yaml))
- `top_choice_prob`: Probability of the most likely topic
- `choice_probs`: Probabilities for all possible topic choices
- `choice_loss`: Average log-likelihood of the answers
- `metadata`: Additional metadata from the source dataset
- `label_permutation`: Permutation used for the label ordering during prompting
- `fewshot_permutation`: Permutation used for few-shot demonstration examples during prompting

## Citation
```bibtex
@article{wettig2025organize,
  title={Organize the Web: Constructing Domains Enhances Pre-Training Data Curation},
  author={Alexander Wettig and Kyle Lo and Sewon Min and Hannaneh Hajishirzi and Danqi Chen and Luca Soldaini},
  journal={arXiv preprint arXiv:2502.10341},
  year={2025}
}
```