Symbolic Techniques for Deep Learning: Challenges and Opportunities
  As the number of deep learning frameworks increase and certain ones gain
popularity, it spurs the discussion of what methodologies are employed by these
frameworks and the reasoning behind them. The goal of this survey is to study
how symbolic techniques are utilized in deep learning. To do this, we look at
some of the most popular deep learning frameworks being used today, including
TensorFlow, Keras, PyTorch, and MXNet. While these frameworks greatly differ
from one another, many of them use symbolic techniques, whether it be symbolic
execution, graphs, or programming. We focus this paper on symbolic techniques
because they influence not only how neural networks are built but also the way
in which they are executed.
  Limitations of symbolic techniques have led to efforts in integrating
symbolic and nonsymbolic aspects in deep learning, opening up new possibilities
for symbolic techniques. For example, the Gluon API by Apache MXNet bridges the
gap between imperative programming and symbolic execution through
hybridization. Frameworks such as JANUS attempt to translate imperative
programs into symbolic graphs, while approaches like DeepCheck attempt to use
symbolic execution to analyze and validate imperative neural network programs.
Symbolic analysis has also been paired with concrete execution in a technique
called concolic testing in order to better test deep neural networks. Our study
of these developments exemplifies just a few of the many ways the symbolic
techniques employed by popular frameworks have the opportunity to be altered
and utilized to achieve better performance.
