Characterizing the Weight Space for Different Learning Models
  Deep Learning has become one of the primary research areas in developing
intelligent machines. Most of the well-known applications (such as Speech
Recognition, Image Processing and NLP) of AI are driven by Deep Learning. Deep
Learning algorithms mimic human brain using artificial neural networks and
progressively learn to accurately solve a given problem. But there are
significant challenges in Deep Learning systems. There have been many attempts
to make deep learning models imitate the biological neural network. However,
many deep learning models have performed poorly in the presence of adversarial
examples. Poor performance in adversarial examples leads to adversarial attacks
and in turn leads to safety and security in most of the applications. In this
paper we make an attempt to characterize the solution space of a deep neural
network in terms of three different subsets viz. weights belonging to exact
trained patterns, weights belonging to generalized pattern set and weights
belonging to adversarial pattern sets. We attempt to characterize the solution
space with two seemingly different learning paradigms viz. the Deep Neural
Networks and the Dense Associative Memory Model, which try to achieve learning
via quite different mechanisms. We also show that adversarial attacks are
generally less successful against Associative Memory Models than Deep Neural
Networks.
