Dither computing: a hybrid deterministic-stochastic computing framework
  Stochastic computing has a long history as an alternative method of
performing arithmetic on a computer. While it can be considered an unbiased
estimator of real numbers, it has a variance and MSE on the order of
$\Omega(\frac{1}{N})$. On the other hand, deterministic variants of stochastic
computing remove the stochastic aspect, but cannot approximate arbitrary real
numbers with arbitrary precision and are biased estimators. However, they have
an asymptotically superior MSE on the order of $O(\frac{1}{N^2})$. Recent
results in deep learning with stochastic rounding suggest that the bias in the
rounding can degrade performance. We proposed an alternative framework, called
dither computing, that combines aspects of stochastic computing and its
deterministic variants and that can perform computing with similar efficiency,
is unbiased, and with a variance and MSE also on the optimal order of
$\Theta(\frac{1}{N^2})$. We also show that it can be beneficial in stochastic
rounding applications as well. We provide implementation details and give
experimental results to comparatively show the benefits of the proposed scheme.
