An implicit function learning approach for parametric modal regression
  For multi-valued functions---such as when the conditional distribution on
targets given the inputs is multi-modal---standard regression approaches are
not always desirable because they provide the conditional mean. Modal
regression algorithms address this issue by instead finding the conditional
mode(s). Most, however, are nonparametric approaches and so can be difficult to
scale. Further, parametric approximators, like neural networks, facilitate
learning complex relationships between inputs and targets. In this work, we
propose a parametric modal regression algorithm. We use the implicit function
theorem to develop an objective, for learning a joint function over inputs and
targets. We empirically demonstrate on several synthetic problems that our
method (i) can learn multi-valued functions and produce the conditional modes,
(ii) scales well to high-dimensional inputs, and (iii) can even be more
effective for certain uni-modal problems, particularly for high-frequency
functions. We demonstrate that our method is competitive in a real-world modal
regression problem and two regular regression datasets.
