Combining 3D Image and Tabular Data via the Dynamic Affine Feature Map
  Transform
  Prior work on diagnosing Alzheimer's disease from magnetic resonance images
of the brain established that convolutional neural networks (CNNs) can leverage
the high-dimensional image information for classifying patients. However,
little research focused on how these models can utilize the usually
low-dimensional tabular information, such as patient demographics or laboratory
measurements. We introduce the Dynamic Affine Feature Map Transform (DAFT), a
general-purpose module for CNNs that dynamically rescales and shifts the
feature maps of a convolutional layer, conditional on a patient's tabular
clinical information. We show that DAFT is highly effective in combining 3D
image and tabular information for diagnosis and time-to-dementia prediction,
where it outperforms competing CNNs with a mean balanced accuracy of 0.622 and
mean c-index of 0.748, respectively. Our extensive ablation study provides
valuable insights into the architectural properties of DAFT. Our implementation
is available at https://github.com/ai-med/DAFT.
