Multi Layer Neural Networks as Replacement for Pooling Operations
  Pooling operations, which can be calculated at low cost and serve as a linear
or nonlinear transfer function for data reduction, are found in almost every
modern neural network. Countless modern approaches have already tackled
replacing the common maximum value selection and mean value operations, not to
mention providing a function that allows different functions to be selected
through changing parameters. Additional neural networks are used to estimate
the parameters of these pooling functions.Consequently, pooling layers may
require supplementary parameters to increase the complexity of the whole model.
In this work, we show that one perceptron can already be used effectively as a
pooling operation without increasing the complexity of the model. This kind of
pooling allows for the integration of multi-layer neural networks directly into
a model as a pooling operation by restructuring the data and, as a result,
learnin complex pooling operations. We compare our approach to tensor
convolution with strides as a pooling operation and show that our approach is
both effective and reduces complexity. The restructuring of the data in
combination with multiple perceptrons allows for our approach to be used for
upscaling, which can then be utilized for transposed convolutions in semantic
segmentation.
