The Helmholtz Method: Using Perceptual Compression to Reduce Machine
  Learning Complexity
  This paper proposes a fundamental answer to a frequently asked question in
multimedia computing and machine learning: Do artifacts from perceptual
compression contribute to error in the machine learning process and if so, how
much? Our approach to the problem is a reinterpretation of the Helmholtz Free
Energy formula from physics to explain the relationship between content and
noise when using sensors (such as cameras or microphones) to capture multimedia
data. The reinterpretation allows a bit-measurement of the noise contained in
images, audio, and video by combining a classifier with perceptual compression,
such as JPEG or MP3. Our experiments on CIFAR-10 as well as Fraunhofer's
IDMT-SMT-Audio-Effects dataset indicate that, at the right quality level,
perceptual compression is actually not harmful but contributes to a significant
reduction of complexity of the machine learning process. That is, our noise
quantification method can be used to speed up the training of deep learning
classifiers significantly while maintaining, or sometimes even improving,
overall classification accuracy. Moreover, our results provide insights into
the reasons for the success of deep learning.
