Aggregated Learning: A Vector-Quantization Approach to Learning Neural
  Network Classifiers
  We consider the problem of learning a neural network classifier. Under the
information bottleneck (IB) principle, we associate with this classification
problem a representation learning problem, which we call "IB learning". We show
that IB learning is, in fact, equivalent to a special class of the quantization
problem. The classical results in rate-distortion theory then suggest that IB
learning can benefit from a "vector quantization" approach, namely,
simultaneously learning the representations of multiple input objects. Such an
approach assisted with some variational techniques, result in a novel learning
framework, "Aggregated Learning", for classification with neural network
models. In this framework, several objects are jointly classified by a single
neural network. The effectiveness of this framework is verified through
extensive experiments on standard image recognition and text classification
tasks.
