Evaluation of data imputation strategies in complex, deeply-phenotyped
  data sets: the case of the EU-AIMS Longitudinal European Autism Project
  An increasing number of large-scale multi-modal research initiatives has been
conducted in the typically developing population, as well as in psychiatric
cohorts. Missing data is a common problem in such datasets due to the
difficulty of assessing multiple measures on a large number of participants.
The consequences of missing data accumulate when researchers aim to explore
relationships between multiple measures. Here we aim to evaluate different
imputation strategies to fill in missing values in clinical data from a large
(total N=764) and deeply characterised (i.e. range of clinical and cognitive
instruments administered) sample of N=453 autistic individuals and N=311
control individuals recruited as part of the EU-AIMS Longitudinal European
Autism Project (LEAP) consortium. In particular we consider a total of 160
clinical measures divided in 15 overlapping subsets of participants. We use two
simple but common univariate strategies, mean and median imputation, as well as
a Round Robin regression approach involving four independent multivariate
regression models including a linear model, Bayesian Ridge regression, as well
as several non-linear models, Decision Trees, Extra Trees and K-Neighbours
regression. We evaluate the models using the traditional mean square error
towards removed available data, and consider in addition the KL divergence
between the observed and the imputed distributions. We show that all of the
multivariate approaches tested provide a substantial improvement compared to
typical univariate approaches. Further, our analyses reveal that across all 15
data-subsets tested, an Extra Trees regression approach provided the best
global results. This allows the selection of a unique model to impute missing
data for the LEAP project and deliver a fixed set of imputed clinical data to
be used by researchers working with the LEAP dataset in the future.
