A generalized quadratic loss for SVM and Deep Neural Networks
  We consider some supervised binary classification tasks and a regression
task, whereas SVM and Deep Learning, at present, exhibit the best
generalization performances. We extend the work [3] on a generalized quadratic
loss for learning problems that examines pattern correlations in order to
concentrate the learning problem into input space regions where patterns are
more densely distributed. From a shallow methods point of view (e.g.: SVM),
since the following mathematical derivation of problem (9) in [3] is incorrect,
we restart from problem (8) in [3] and we try to solve it with one procedure
that iterates over the dual variables until the primal and dual objective
functions converge. In addition we propose another algorithm that tries to
solve the classification problem directly from the primal problem formulation.
We make also use of Multiple Kernel Learning to improve generalization
performances. Moreover, we introduce for the first time a custom loss that
takes in consideration pattern correlation for a shallow and a Deep Learning
task. We propose some pattern selection criteria and the results on 4 UCI
data-sets for the SVM method. We also report the results on a larger binary
classification data-set based on Twitter, again drawn from UCI, combined with
shallow Learning Neural Networks, with and without the generalized quadratic
loss. At last, we test our loss with a Deep Neural Network within a larger
regression task taken from UCI. We compare the results of our optimizers with
the well known solver SVMlight and with Keras Multi-Layers Neural Networks with
standard losses and with a parameterized generalized quadratic loss, and we
obtain comparable results.
