Differentiable Projection for Constrained Deep Learning
  Deep neural networks (DNNs) have achieved extraordinary performance in
solving different tasks in various fields. However, the conventional DNN model
is steadily approaching the ground-truth value through loss backpropagation. In
some applications, some prior knowledge could be easily obtained, such as
constraints which the ground truth observation follows. Here, we try to give a
general approach to incorporate information from these constraints to enhance
the performance of the DNNs. Theoretically, we could formulate these kinds of
problems as constrained optimization problems that KKT conditions could solve.
In this paper, we propose to use a differentiable projection layer in DNN
instead of directly solving time-consuming KKT conditions. The proposed
projection method is differentiable, and no heavy computation is required.
Finally, we also conducted some experiments using a randomly generated
synthetic dataset and image segmentation task using the PASCAL VOC dataset to
evaluate the performance of the proposed projection method. Experimental
results show that the projection method is sufficient and outperforms baseline
methods.
