MiniF2F: a cross-system benchmark for formal Olympiad-level mathematics
  We present miniF2F, a dataset of formal Olympiad-level mathematics problems
statements intended to provide a unified cross-system benchmark for neural
theorem proving. The miniF2F benchmark currently targets Metamath, Lean,
Isabelle (partially) and HOL Light (partially) and consists of 488 problem
statements drawn from the AIME, AMC, and the International Mathematical
Olympiad (IMO), as well as material from high-school and undergraduate
mathematics courses. We report baseline results using GPT-f, a neural theorem
prover based on GPT-3 and provide an analysis of its performance. We intend for
miniF2F to be a community-driven effort and hope that our benchmark will help
spur advances in neural theorem proving.
