Beyond Application End-Point Results: Quantifying Statistical Robustness
  of MCMC Accelerators
  Statistical machine learning often uses probabilistic algorithms, such as
Markov Chain Monte Carlo (MCMC), to solve a wide range of problems.
Probabilistic computations, often considered too slow on conventional
processors, can be accelerated with specialized hardware by exploiting
parallelism and optimizing the design using various approximation techniques.
Current methodologies for evaluating correctness of probabilistic accelerators
are often incomplete, mostly focusing only on end-point result quality
("accuracy"). It is important for hardware designers and domain experts to look
beyond end-point "accuracy" and be aware of the hardware optimizations impact
on other statistical properties.
  This work takes a first step towards defining metrics and a methodology for
quantitatively evaluating correctness of probabilistic accelerators beyond
end-point result quality. We propose three pillars of statistical robustness:
1) sampling quality, 2) convergence diagnostic, and 3) goodness of fit. We
apply our framework to a representative MCMC accelerator and surface design
issues that cannot be exposed using only application end-point result quality.
Applying the framework to guide design space exploration shows that statistical
robustness comparable to floating-point software can be achieved by slightly
increasing the bit representation, without floating-point hardware
requirements.
