Adversarial Counterfactual Augmentation: Application in Alzheimer's
  Disease Classification
  Data augmentation has been widely used in deep learning to reduce
over-fitting and improve the robustness of models. However, traditional data
augmentation techniques, e.g., rotation, cropping, flipping, etc., do not
consider \textit{semantic} transformations, e.g., changing the age of a brain
image. Previous works tried to achieve semantic augmentation by generating
\textit{counterfactuals}, but they focused on how to train deep generative
models and randomly created counterfactuals with the generative models without
considering which counterfactuals are most \textit{effective} for improving
downstream training. Different from these approaches, in this work, we propose
a novel adversarial counterfactual augmentation scheme that aims to find the
most \textit{effective} counterfactuals to improve downstream tasks with a
pre-trained generative model. Specifically, we construct an adversarial game
where we update the input \textit{conditional factor} of the generator and the
downstream \textit{classifier} with gradient backpropagation alternatively and
iteratively. The key idea is to find conditional factors that can result in
\textit{hard} counterfactuals for the classifier. This can be viewed as finding
the `\textit{weakness}' of the classifier and purposely forcing it to
\textit{overcome} its weakness via the generative model. To demonstrate the
effectiveness of the proposed approach, we validate the method with the
classification of Alzheimer's Disease (AD) as the downstream task based on a
pre-trained brain ageing synthesis model. We show the proposed approach
improves test accuracy and can alleviate spurious correlations. Code will be
released upon acceptance.
