task_categories:token-classification language:en size_categories:100K<n<1M format:parquet modality:tabular modality:text library:datasets library:pandas library:mlcroissant library:polars arxiv:2411.14073 region:us physics astrophysics high energy physics science Wikipedia
---
task_categories:
- token-classification
language:
- en
tags:
- physics
- astrophysics
- high energy physics
- science
- Wikipedia
pretty_name: Wikipedia-Physics Corpus
size_categories:
- 10K<n<100K
---

# Dataset Card for the Wikipedia-Physics Corpus

**Wikipedia-Physics Corpus** contains 102,409 paragraphs extracted from 6,642 key physics-related Wikipedia articles as well as word-sense labels for 1,186 occurrences of "Planck" across 885 paragraphs.

This table details the main columns of the dataset:

|Column|Description|
|:----------:|:-:|
|*text*|Full text of the paragraph|
|*length*|Number of unicode characters in the paragraph|
|*page_title*|Titel of the Wikipedia article|
|*url*|URL of the Wikipedia article|
|*text_id*|ID of the text/paragraph|
|*paragraph_idx*|Relative index of the text/paragraph|
|*year*|Year of the download of the Wikipedia article|
|*month*|Month of the download of the Wikipedia article|
|*day*|Day of the download of the Wikipedia article|
|*minute*|Year of the download of the Wikipedia article|
|*second*|Year of the download of the Wikipedia article|
|*num_planck_labels*|Number of labeled occurrences of "Planck" in the paragraph|
|*planck_labels*|List of word-sense labels for all occurrences of "Planck" in the paragraph|

The primary purpose of the Wikipedia-Physics Corpus lies in its use for analyzing the meaning of concepts in physics. For further insights into the corpus and the underlying research project (<a target="_blank" rel="noopener noreferrer" href="https://doi.org/10.3030/101044932" >Network Epistemology in Practice</a>) please refer to this paper:

- <a target="_blank" rel="noopener noreferrer" href="https://arxiv.org/abs/2411.14073">Simons, A. (2024). Meaning at the planck scale? Contextualized word embeddings for doing history, philosophy, and sociology of science. arXiv:2411.14073.</a>


## Construction

Wikipedia articles were selected using the <a target="_blank" rel="noopener noreferrer" href="https://petscan.wmcloud.org/">PetScan</a> tool, which generated a list of all pages categorized under "physics" or its immediate subcategories. Markup was removed and minimal cleaning applied to produce plain text paragraphs, and while references were removed, all formulas were retained.

Additionally, 1,186 occurrences of the term "Planck" were labeled across 885 paragraphs containing the term. These occurrences were identified using a case-insensitive regular expression to capture variations like "PLANCK" and "Planck(2015)"" while excluding irrelevant forms like "planckian". The labels denote distinct meanings of the target term, such as PERSON, CONSTANT, UNITS, LAW, MPS, MISSION, and FOKKER.

## Details

- **Developer:** <a target="_blank" rel="noopener noreferrer" href="https://www.tu.berlin/en/hps-mod-sci/arno-simons">Arno Simons</a>
- **Funded by:** The European Union under Grant agreement ID: <a target="_blank" rel="noopener noreferrer" href="https://doi.org/10.3030/101044932" >101044932</a>
- **Language:** English