Masked Image Modeling Advances 3D Medical Image Analysis
  Recently, masked image modeling (MIM) has gained considerable attention due
to its capacity to learn from vast amounts of unlabeled data and has been
demonstrated to be effective on a wide variety of vision tasks involving
natural images. Meanwhile, the potential of self-supervised learning in
modeling 3D medical images is anticipated to be immense due to the high
quantities of unlabeled images, and the expense and difficulty of quality
labels. However, MIM's applicability to medical images remains uncertain. In
this paper, we demonstrate that masked image modeling approaches can also
advance 3D medical images analysis in addition to natural images. We study how
masked image modeling strategies leverage performance from the viewpoints of 3D
medical image segmentation as a representative downstream task: i) when
compared to naive contrastive learning, masked image modeling approaches
accelerate the convergence of supervised training even faster (1.40$\times$)
and ultimately produce a higher dice score; ii) predicting raw voxel values
with a high masking ratio and a relatively smaller patch size is non-trivial
self-supervised pretext-task for medical images modeling; iii) a lightweight
decoder or projection head design for reconstruction is powerful for masked
image modeling on 3D medical images which speeds up training and reduce cost;
iv) finally, we also investigate the effectiveness of MIM methods under
different practical scenarios where different image resolutions and labeled
data ratios are applied.
