task_categories:text2text-generation language:en size_categories:10K<n<100K format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
task_categories:
- text2text-generation
language:
- en
pretty_name: Open Reviews Summary on ML Paper's Abstract
size_categories:
- 10K<n<100K
---

# Dataset Description
## Abstract
The Abstract Paper Reviews Dataset is designed for training machine learning models to generate reviews of academic papers based on the paper's title and abstract. It is formatted in a conversational style, facilitating direct use with models like the SFTTrainer without the need for additional parsing or conversion into a chat template. This dataset enables the development of models that can assist in peer review processes by providing preliminary feedback on academic papers.

## Languages
The text in the dataset is primarily in English.

## Example
```json
{
  "messages": [
      {
          "content": "You are a professional machine learning conference reviewer who reviews a given paper and considers 4 criteria: [Significance and novelty], [Potential reasons for acceptance], [Potential reasons for rejection], and [Suggestions for improvement]. For each criterion, provide random number of supporting points derived from the paper's content. And for each supporting point, use the format: '<title of supporting point>' followed by a detailed explanation. Your response should only include your reviews only, which means always start with [Significance and novelty], dont' repeat the given paper and output things other than your reviews in required format. The paper is given as follows:",
          "role": "system"
      },
      {
          "content": "[TITLE]\nOn the optimal precision of GANs\n\n[ABSTRACT]\nGenerative adversarial networks (GANs) are known to face model misspecification when learning disconnected distributions. Indeed, continuous mapping from a unimodal latent distribution to a disconnected one is impossible, so GANs necessarily generate samples outside of the support of the target distribution. In this paper, we make the connection between the performance of GANs and their latent space configuration. In particular, we raise the following question: what is the latent space partition that minimizes the measure of out-of-manifold samples? Building on a recent result of geometric measure theory, we prove there exist optimal GANs when the dimension of the latent space is larger than the number of modes. In particular, we show that these generators structure their latent space as a `simplicial cluster' - a Voronoi partition where centers are equally distant.  We derive both an upper and a lower bound on the optimal precision of GANs learning disconnected manifolds. Interestingly, these two bounds have the same order of decrease: $\\sqrt{\\log m}$, $m$ being the number of modes. Finally, we perform several experiments to exhibit the geometry of the latent space and experimentally show that GANs have a geometry with similar properties to the theoretical one.",
          "role": "user"
      },
      {
          "content": "[Significance and novelty]\n<Insights on GANs using Gaussian isoperimetry> The paper brings in a novel perspective by using Gaussian isoperimetry to explore GAN representations and sheds light onto the key aspects of GANs, from mode collapse to their precision\/recall performance, especially in the case of disconnected distributions.\n<Exploration of latent space organization for GANs> The exploration of latent space organization for GANs and how it affects precision is a significant and somewhat new contribution to the field of generative models and machine learning.\n\n[Potential reasons for acceptance]\n<Novel theoretical perspective> The paper offers a novel theoretical perspective on GANs, using results from Gaussian isoperimetry to analyze the precision of GANs under certain circumstances.\n<Insights for future research> The paper provides theoretical and empirical insights that can guide future research in improving GAN precision and exploring latent space organization.\n\n[Potential reasons for rejection]\n<Unclear relevance to real data setting> The paper lacks clarity on how the theoretical findings relate to real-world data, which impedes the practical impact of the research.\n<Limitations in practical application> The paper's limitations in connecting theoretical insights to practical applications may warrant further revisions before acceptance.\n\n[Suggestions for improvement]\n<Clarify relevance to real-world data> Further clarity is needed on how the theoretical results can be practically applied to real-world datasets and how the assumptions align with the complexities of real data.\n<Enhance empirical validation> The paper might benefit from strengthening the connection between the theoretical insights and empirical validation, especially in exploring the relevance of assumptions to practical scenarios.\n\n",
          "role": "assistant"
      }
  ]
}
```

## Size
Training dataset size: **12452** \
Testing dataset size: **3113**

## Curation Rationale
The dataset was curated to provide a specialized tool for training models capable of generating constructive feedback on academic papers, specifically in the field of machine learning. This can aid in automating aspects of the peer review process and assist researchers in refining their work before submission.

## Source Data
The papers and their reviews in this dataset are sourced from OpenReview. The reviews are summaries generated by GPT-4, which synthesizes insights from multiple human-written reviews for each paper. This approach ensures a comprehensive representation of the diverse opinions and critiques expressed by the academic community, while maintaining consistency and conciseness in the review content.