Convolutional Transformer based Dual Discriminator Generative
  Adversarial Networks for Video Anomaly Detection
  Detecting abnormal activities in real-world surveillance videos is an
important yet challenging task as the prior knowledge about video anomalies is
usually limited or unavailable. Despite that many approaches have been
developed to resolve this problem, few of them can capture the normal
spatio-temporal patterns effectively and efficiently. Moreover, existing works
seldom explicitly consider the local consistency at frame level and global
coherence of temporal dynamics in video sequences. To this end, we propose
Convolutional Transformer based Dual Discriminator Generative Adversarial
Networks (CT-D2GAN) to perform unsupervised video anomaly detection.
Specifically, we first present a convolutional transformer to perform future
frame prediction. It contains three key components, i.e., a convolutional
encoder to capture the spatial information of the input video clips, a temporal
self-attention module to encode the temporal dynamics, and a convolutional
decoder to integrate spatio-temporal features and predict the future frame.
Next, a dual discriminator based adversarial training procedure, which jointly
considers an image discriminator that can maintain the local consistency at
frame-level and a video discriminator that can enforce the global coherence of
temporal dynamics, is employed to enhance the future frame prediction. Finally,
the prediction error is used to identify abnormal video frames. Thoroughly
empirical studies on three public video anomaly detection datasets, i.e., UCSD
Ped2, CUHK Avenue, and Shanghai Tech Campus, demonstrate the effectiveness of
the proposed adversarial spatio-temporal modeling framework.
