Sequential Anomaly Detection using Inverse Reinforcement Learning
  One of the most interesting application scenarios in anomaly detection is
when sequential data are targeted. For example, in a safety-critical
environment, it is crucial to have an automatic detection system to screen the
streaming data gathered by monitoring sensors and to report abnormal
observations if detected in real-time. Oftentimes, stakes are much higher when
these potential anomalies are intentional or goal-oriented. We propose an
end-to-end framework for sequential anomaly detection using inverse
reinforcement learning (IRL), whose objective is to determine the
decision-making agent's underlying function which triggers his/her behavior.
The proposed method takes the sequence of actions of a target agent (and
possibly other meta information) as input. The agent's normal behavior is then
understood by the reward function which is inferred via IRL. We use a neural
network to represent a reward function. Using a learned reward function, we
evaluate whether a new observation from the target agent follows a normal
pattern. In order to construct a reliable anomaly detection method and take
into consideration the confidence of the predicted anomaly score, we adopt a
Bayesian approach for IRL. The empirical study on publicly available real-world
data shows that our proposed method is effective in identifying anomalies.
