Follow the bisector: a simple method for multi-objective optimization
  This study presents a novel Equiangular Direction Method (EDM) to solve a
multi-objective optimization problem. We consider optimization problems, where
multiple differentiable losses have to be minimized. The presented method
computes descent direction in every iteration to guarantee equal relative
decrease of objective functions. This descent direction is based on the
normalized gradients of the individual losses. Therefore, it is appropriate to
solve multi-objective optimization problems with multi-scale losses. We test
the proposed method on the imbalanced classification problem and multi-task
learning problem, where standard datasets are used. EDM is compared with other
methods to solve these problems.
