The Medical Segmentation Decathlon
  International challenges have become the de facto standard for comparative
assessment of image analysis algorithms given a specific task. Segmentation is
so far the most widely investigated medical image processing task, but the
various segmentation challenges have typically been organized in isolation,
such that algorithm development was driven by the need to tackle a single
specific clinical problem. We hypothesized that a method capable of performing
well on multiple tasks will generalize well to a previously unseen task and
potentially outperform a custom-designed solution. To investigate the
hypothesis, we organized the Medical Segmentation Decathlon (MSD) - a
biomedical image analysis challenge, in which algorithms compete in a multitude
of both tasks and modalities. The underlying data set was designed to explore
the axis of difficulties typically encountered when dealing with medical
images, such as small data sets, unbalanced labels, multi-site data and small
objects. The MSD challenge confirmed that algorithms with a consistent good
performance on a set of tasks preserved their good average performance on a
different set of previously unseen tasks. Moreover, by monitoring the MSD
winner for two years, we found that this algorithm continued generalizing well
to a wide range of other clinical problems, further confirming our hypothesis.
Three main conclusions can be drawn from this study: (1) state-of-the-art image
segmentation algorithms are mature, accurate, and generalize well when
retrained on unseen tasks; (2) consistent algorithmic performance across
multiple tasks is a strong surrogate of algorithmic generalizability; (3) the
training of accurate AI segmentation models is now commoditized to non AI
experts.
