Anomaly Detection Under Controlled Sensing Using Actor-Critic
  Reinforcement Learning
  We consider the problem of detecting anomalies among a given set of processes
using their noisy binary sensor measurements. The noiseless sensor measurement
corresponding to a normal process is 0, and the measurement is 1 if the process
is anomalous. The decision-making algorithm is assumed to have no knowledge of
the number of anomalous processes. The algorithm is allowed to choose a subset
of the sensors at each time instant until the confidence level on the decision
exceeds the desired value. Our objective is to design a sequential sensor
selection policy that dynamically determines which processes to observe at each
time and when to terminate the detection algorithm. The selection policy is
designed such that the anomalous processes are detected with the desired
confidence level while incurring minimum cost which comprises the delay in
detection and the cost of sensing. We cast this problem as a sequential
hypothesis testing problem within the framework of Markov decision processes,
and solve it using the actor-critic deep reinforcement learning algorithm. This
deep neural network-based algorithm offers a low complexity solution with good
detection accuracy. We also study the effect of statistical dependence between
the processes on the algorithm performance. Through numerical experiments, we
show that our algorithm is able to adapt to any unknown statistical dependence
pattern of the processes.
