UNet++: Redesigning Skip Connections to Exploit Multiscale Features in
  Image Segmentation
  The state-of-the-art models for medical image segmentation are variants of
U-Net and fully convolutional networks (FCN). Despite their success, these
models have two limitations: (1) their optimal depth is apriori unknown,
requiring extensive architecture search or inefficient ensemble of models of
varying depths; and (2) their skip connections impose an unnecessarily
restrictive fusion scheme, forcing aggregation only at the same-scale feature
maps of the encoder and decoder sub-networks. To overcome these two
limitations, we propose UNet++, a new neural architecture for semantic and
instance segmentation, by (1) alleviating the unknown network depth with an
efficient ensemble of U-Nets of varying depths, which partially share an
encoder and co-learn simultaneously using deep supervision; (2) redesigning
skip connections to aggregate features of varying semantic scales at the
decoder sub-networks, leading to a highly flexible feature fusion scheme; and
(3) devising a pruning scheme to accelerate the inference speed of UNet++. We
have evaluated UNet++ using six different medical image segmentation datasets,
covering multiple imaging modalities such as computed tomography (CT), magnetic
resonance imaging (MRI), and electron microscopy (EM), and demonstrating that
(1) UNet++ consistently outperforms the baseline models for the task of
semantic segmentation across different datasets and backbone architectures; (2)
UNet++ enhances segmentation quality of varying-size objects -- an improvement
over the fixed-depth U-Net; (3) Mask RCNN++ (Mask R-CNN with UNet++ design)
outperforms the original Mask R-CNN for the task of instance segmentation; and
(4) pruned UNet++ models achieve significant speedup while showing only modest
performance degradation. Our implementation and pre-trained models are
available at https://github.com/MrGiovanni/UNetPlusPlus.
