language:en license:mit size_categories:10K<n<100K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: id
    dtype: int64
  - name: input
    dtype: string
  - name: output
    dtype: string
  splits:
  - name: train
    num_bytes: 21642133
    num_examples: 10000
  - name: validation
    num_bytes: 6230302
    num_examples: 3000
  - name: test
    num_bytes: 6255318
    num_examples: 3000
  download_size: 17469502
  dataset_size: 34127753
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
  - split: validation
    path: data/validation-*
  - split: test
    path: data/test-*
license: mit
language:
- en
---
# Dataset Card for *Scientific Papers - Cleaned*

This dataset card provides information about the *Scientific Papers - Cleaned* dataset, a refined and curated version of scientific papers for research and natural language processing tasks.

## Dataset Details

### Dataset Description

The *Scientific Papers - Cleaned* dataset is a cleaned version of a larger collection of scientific papers. It is designed to provide structured input-output pairs for tasks such as text summarization, paraphrasing, or scientific language understanding. This dataset includes unique identifiers, input texts (e.g., abstracts or sections of papers), and corresponding output texts (e.g., summaries or rewritten versions).

- **Curated by:** [Wilbert Chandra (Username: Gilbert Krantzx)](https://huggingface.co/GilbertKrantzx)
- **Language(s) (NLP):** English
- **License:** MIT License


### Dataset Sources

- **Original Repository:** [Hugging Face](https://huggingface.co/datasets/scillm/scientific_papers-archive)

## Uses

### Direct Use

The dataset can be used for various natural language processing tasks, including but not limited to:

- Text summarization
- Language modeling for scientific contexts
- Textual entailment and paraphrase detection
- Fine-tuning language models for scientific writing

### Out-of-Scope Use

The dataset may not be suitable for:

- Applications involving real-time decision-making systems without validation
- Use cases where data sensitivity or misrepresentation of scientific context could lead to harm

## Dataset Structure

The dataset consists of three splits: **train**, **validation**, and **test**.

### Features

| **Feature** | **Type**   | **Description**                                  |
|-------------|------------|--------------------------------------------------|
| `id`        | `int64`    | Unique identifier for each record                |
| `input`     | `string`   | Input text, such as a section or paragraph       |
| `output`    | `string`   | Corresponding output text, such as a summary     |

### Splits

| **Split**      | **Size (bytes)** | **Examples** |
|----------------|------------------|--------------|
| `train`        | 21,642,133       | 10,000       |
| `validation`   | 6,230,302        | 3,000        |
| `test`         | 6,255,318        | 3,000        |

### Dataset Statistics

- **Download Size:** 17.5 MB
- **Dataset Size:** 34.1 MB

## Dataset Creation

### Curation Rationale

The dataset was created to facilitate research in text generation and summarization for scientific texts, emphasizing clarity and conciseness.

### Source Data

#### Data Collection and Processing

The data originates from the [Scientific Papers Archive](https://huggingface.co/datasets/scillm/scientific_papers-archive). The original data has been cleaned and standardized to enhance usability, with processes applied to remove noise, standardize formats, and ensure quality.

#### Who are the source data producers?

The dataset is derived from scientific publications, with content generated by researchers, scientists, and academic professionals.

## Bias, Risks, and Limitations

The dataset may contain biases inherent in its source material, such as overrepresentation of specific scientific disciplines or regions.

### Recommendations

Users should exercise caution and validate findings, particularly in interdisciplinary or applied research contexts.

## Citation

**BibTeX:**
```bibtex
@dataset{scientific_papers_cleaned,
  author = {Wilbert Chandra (Username: Gilbert Krantzx)},
  title = {Scientific Papers - Cleaned},
  year = {2024},
  howpublished = {Hugging Face Dataset Repository},
  url = {https://huggingface.co/GilbertKrantzx/scientific_papers-cleaned}
}
```

**APA:**
Wilbert Chandra (Username: Gilbert Krantzx). (2024). *Scientific Papers - Cleaned*. Retrieved from [https://huggingface.co/GilbertKrantzx/scientific_papers-cleaned](https://huggingface.co/GilbertKrantzx/scientific_papers-cleaned)

## Dataset Card Contact

For inquiries, contact Wilbert Chandra (Username: Gilbert Krantzx) via the Hugging Face repository.

Let me know if this works!