Learning Energy-Based Approximate Inference Networks for Structured
  Applications in NLP
  Structured prediction in natural language processing (NLP) has a long
history. The complex models of structured application come at the difficulty of
learning and inference. These difficulties lead researchers to focus more on
models with simple structure components (e.g., local classifier). Deep
representation learning has become increasingly popular in recent years. The
structure components of their method, on the other hand, are usually relatively
simple. We concentrate on complex structured models in this dissertation. We
provide a learning framework for complicated structured models as well as an
inference method with a better speed/accuracy/search error trade-off. The
dissertation begins with a general introduction to energy-based models. In NLP
and other applications, an energy function is comparable to the concept of a
scoring function. In this dissertation, we discuss the concept of the energy
function and structured models with different energy functions. Then, we
propose a method in which we train a neural network to do argmax inference
under a structured energy function, referring to the trained networks as
"inference networks" or "energy-based inference networks". We then develop ways
of jointly learning energy functions and inference networks using an
adversarial learning framework. Despite the inference and learning difficulties
of energy-based models, we present approaches in this thesis that enable
energy-based models more easily to be applied in structured NLP applications.
