Meta-AAD: Active Anomaly Detection with Deep Reinforcement Learning
  High false-positive rate is a long-standing challenge for anomaly detection
algorithms, especially in high-stake applications. To identify the true
anomalies, in practice, analysts or domain experts will be employed to
investigate the top instances one by one in a ranked list of anomalies
identified by an anomaly detection system. This verification procedure
generates informative labels that can be leveraged to re-rank the anomalies so
as to help the analyst to discover more true anomalies given a time budget.
Some re-ranking strategies have been proposed to approximate the above
sequential decision process. Specifically, existing strategies have been
focused on making the top instances more likely to be anomalous based on the
feedback. Then they greedily select the top-1 instance for query. However,
these greedy strategies could be sub-optimal since some low-ranked instances
could be more helpful in the long-term. In this work, we propose Active Anomaly
Detection with Meta-Policy (Meta-AAD), a novel framework that learns a
meta-policy for query selection. Specifically, Meta-AAD leverages deep
reinforcement learning to train the meta-policy to select the most proper
instance to explicitly optimize the number of discovered anomalies throughout
the querying process. Meta-AAD is easy to deploy since a trained meta-policy
can be directly applied to any new datasets without further tuning. Extensive
experiments on 24 benchmark datasets demonstrate that Meta-AAD significantly
outperforms the state-of-the-art re-ranking strategies and the unsupervised
baseline. The empirical analysis shows that the trained meta-policy is
transferable and inherently achieves a balance between long-term and short-term
rewards.
