Differentiable programming: Generalization, characterization and
  limitations of deep learning
  In the past years, deep learning models have been successfully applied in
several cognitive tasks. Originally inspired by neuroscience, these models are
specific examples of differentiable programs. In this paper we define and
motivate differentiable programming, as well as specify some program
characteristics that allow us to incorporate the structure of the problem in a
differentiable program. We analyze different types of differentiable programs,
from more general to more specific, and evaluate, for a specific problem with a
graph dataset, its structure and knowledge with several differentiable programs
using those characteristics. Finally, we discuss some inherent limitations of
deep learning and differentiable programs, which are key challenges in
advancing artificial intelligence, and then analyze possible solutions
