task_categories:image-to-text language:en license:cc-by-4.0 arxiv:2407.04903 region:us nature nature communications nature sciences multi-discipline multi-modal science understanding large vision language models multimodal large language models
---
license: cc-by-4.0
task_categories:
- image-to-text
language:
- en
pretty_name: NatureCommsCorpus
size_categories:
- 100G<n<1000G
tags:
- nature
- nature communications
- nature sciences
- multi-discipline
- multi-modal
- science understanding
- large vision language models
- multimodal large language models
---

## Dataset Structure

The dataset is stored in the `rawdata` directory, organized into multiple levels for ease of access and categorization. Below is a detailed explanation of the directory structure:

### First-Level Category Directories
These directories represent the five major categories within our dataset:
- **Biological sciences**
- **Earth and environmental sciences**
- **Health sciences**
- **Physical sciences**
- **Scientific community and society**

### Second-Level Subject Directories
Each category contains directories representing various subjects within that field.

### Third-Level Article Directories
Under each subject, there are multiple directories, each named with a unique article ID. These directories represent individual articles and contain the following:

- **Images of Figures**: All figures from the respective article.
- **Metadata File**: `{uid}_data.json` file, which includes information about the article such as title, abstract, figures, and references.

### Metadata Details (`{uid}_data.json`)
The `{uid}_data.json` file contains the following keys:

- `pdf_link`: Link to the PDF of the article (not included in the dataset).
- `review_pdf_link`: Link to the paper review PDF, if available.
- `unique_id`: Unique ID of the article, matching the directory name.
- `images`: Contains information about the figures in the article. Each figure includes:
  - `image_filename`: Filename of the image within the directory.
  - `text_filename`: Filename of the text file containing the figure's caption.
  - `caption`: Main caption for the figure.
  - `description`: Detailed caption, including sub-captions.
- `title`: Title of the article.
- `abstract`: Abstract of the article.
- `published_time`: Publication date of the article.
- `sections`: Lists all sections of the main article content. Each section includes:
  - `section`: Name of the section.
  - `content`: Content of the section.
- `references`: List of references cited in the article. Each reference includes:
  - `idx`: Index of the reference in the article.
  - `title`: Title of the reference.
  - `link`: Link to the reference.

### Example Metadata File
Below is an example of the structure of a metadata JSON file:

```json
{
  "pdf_link": "https://www.nature.com/articles/ncomms1240.pdf",
  "review_pdf_link": "URL not found",
  "unique_id": "ncomms1240",
  "images": [
    {
      "image_filename": "figure_0.png",
      "text_filename": "figure_0_info.txt",
      "caption": "Figure 1: 5hmC preferentially appears in the paternal genome of early mouse preimplantation embryos.",
      "description": "(a) Representative images of mid PN3 and metaphase stage zygotes stained with 5-methylcytosine (5mC, green mouse monoclonal, gift from Dirk Sch\u00fcbeler) and 5-hydroxymethylcytosine (5hmC, red rabbit polyclonal from Active Motif) antibodies. (b) Dynamic appearance of 5hmC during early preimplantation development. Shown are representative images of embryos stained with DNA (blue mouse monoclonal from Millipore) and 5hmC (red rabbit polyclonal from Active Motif) antibodies. (c) Quantification of 5hmC signal normalized against DNA antibody signal. A total of 12\u201318 precisely staged embryos per pronuclear stage from 3 to 5 in vitro fertilization (IVF) experiments were analysed..."
    }
  ],
  "title": "5-Hydroxymethylcytosine in the mammalian zygote is linked with epigenetic reprogramming",
  "published_time": "2011-03-15",
  "abstract": "The epigenomes of early mammalian embryos are extensively reprogrammed to acquire a totipotent developmental potential. A major initial event in this reprogramming is the active loss/demethylation of 5-methylcytosine (5mC) in the zygote. Here, we report on findings that link this active demethylation to molecular mechanisms. We detect 5-hydroxymethylcytosine (5hmC) as a novel modification in mouse, bovine and rabbit zygotes...",
  "sections": [
    {
      "section": "Introduction",
      "content": "Genome-wide reprogramming of DNA methylation (5-methylcytosine; 5mC) is an important epigenomic process observed in mammalian primordial germ cells and early embryos. The reprogramming of DNA methylation has a direct influence on genomic imprints, the regulation of pluripotency and stem cell networks, the erasure of epimutations and the transcriptional control of transposons [1], [2], [3]. In the mammalian zygote, the 5mC content of the paternal chromosomes is substantially reduced before replication of DNA, as detected by immunofluorescence (IF) imaging with antibodies against 5mC [4], [5], [6], [7]. Several lines of evidence suggest that the conversion of 5mC to cytosine can be induced by deamination followed by DNA glycosylase-induced base excision repair [8], [9], [10]. We recently analysed the dynamics of active loss/demethylation of 5mC using bisulphite sequencing on staged mouse zygotes. We could indeed find a potential link between 5mC demethylation and base excision repair..."
    }
  ],
  "references": [
    {
      "idx": "1",
      "title": "Reik, W., Dean, W. & Walter, J. Epigenetic reprogramming in mammalian development. Science 293, 1089\u20131093 (2001).",
      "link": "https://doi.org/10.1126%2Fscience.1063443"
    }
  ]
}
```

## Usage Note

The dataset includes various metadata for scientific articles, including abstracts, figure captions, and references, which can be used for tasks such as article summarization, figure captioning, or reference extraction. The directory structure allows researchers to access data at different levels, depending on the task.

## Additional Information

### Licensing Information

This corpus is released under [CC BY-NC-SA 4.0](http://creativecommons.org/licenses/by-nc-sa/4.0/).

### Citation Information

```
@article{li2024mmsci,
  title={Mmsci: A multimodal multi-discipline dataset for phd-level scientific comprehension},
  author={Li, Zekun and Yang, Xianjun and Choi, Kyuri and Zhu, Wanrong and Hsieh, Ryan and Kim, HyeonJung and Lim, Jin Hyuk and Ji, Sungyoung and Lee, Byungju and Yan, Xifeng and others},
  journal={arXiv preprint arXiv:2407.04903},
  year={2024}
}
```