Improving Neural Network Learning Through Dual Variable Learning Rates
  This paper introduces and evaluates a novel training method for neural
networks: Dual Variable Learning Rates (DVLR). Building on insights from
behavioral psychology, the dual learning rates are used to emphasize correct
and incorrect responses differently, thereby making the feedback to the network
more specific. Further, the learning rates are varied as a function of the
network's performance, thereby making it more efficient. DVLR was implemented
on three types of networks: feedforward, convolutional, and residual, and two
domains: MNIST and CIFAR-10. The results suggest a consistently improved
accuracy, demonstrating that DVLR is a promising, psychologically motivated
technique for training neural network models.
