HPC AI500: A Benchmark Suite for HPC AI Systems
  In recent years, with the trend of applying deep learning (DL) in high
performance scientific computing, the unique characteristics of emerging DL
workloads in HPC raise great challenges in designing, implementing HPC AI
systems. The community needs a new yard stick for evaluating the future HPC
systems. In this paper, we propose HPC AI500 --- a benchmark suite for
evaluating HPC systems that running scientific DL workloads. Covering the most
representative scientific fields, each workload from HPC AI500 is based on
real-world scientific DL applications. Currently, we choose 14 scientific DL
benchmarks from perspectives of application scenarios, data sets, and software
stack. We propose a set of metrics for comprehensively evaluating the HPC AI
systems, considering both accuracy, performance as well as power and cost. We
provide a scalable reference implementation of HPC AI500. HPC AI500 is a part
of the open-source AIBench project, the specification and source code are
publicly available from \url{http://www.benchcouncil.org/AIBench/index.html}.
