ASAC: Active Sensing using Actor-Critic models
  Deciding what and when to observe is critical when making observations is
costly. In a medical setting where observations can be made sequentially,
making these observations (or not) should be an active choice. We refer to this
as the active sensing problem. In this paper, we propose a novel deep learning
framework, which we call ASAC (Active Sensing using Actor-Critic models) to
address this problem. ASAC consists of two networks: a selector network and a
predictor network. The selector network uses previously selected observations
to determine what should be observed in the future. The predictor network uses
the observations selected by the selector network to predict a label, providing
feedback to the selector network (well-selected variables should be predictive
of the label). The goal of the selector network is then to select variables
that balance the cost of observing the selected variables with their predictive
power; we wish to preserve the conditional label distribution. During training,
we use the actor-critic models to allow the loss of the selector to be
"back-propagated" through the sampling process. The selector network "acts" by
selecting future observations to make. The predictor network acts as a "critic"
by feeding predictive errors for the selected variables back to the selector
network. In our experiments, we show that ASAC significantly outperforms
state-of-the-arts in two real-world medical datasets.
