PTNet: A High-Resolution Infant MRI Synthesizer Based on Transformer
  Magnetic resonance imaging (MRI) noninvasively provides critical information
about how human brain structures develop across stages of life. Developmental
scientists are particularly interested in the first few years of
neurodevelopment. Despite the success of MRI collection and analysis for
adults, it is a challenge for researchers to collect high-quality multimodal
MRIs from developing infants mainly because of their irregular sleep pattern,
limited attention, inability to follow instructions to stay still, and a lack
of analysis approaches. These challenges often lead to a significant reduction
of usable data. To address this issue, researchers have explored various
solutions to replace corrupted scans through synthesizing realistic MRIs. Among
them, the convolution neural network (CNN) based generative adversarial network
has demonstrated promising results and achieves state-of-the-art performance.
However, adversarial training is unstable and may need careful tuning of
regularization terms to stabilize the training. In this study, we introduced a
novel MRI synthesis framework - Pyramid Transformer Net (PTNet). PTNet consists
of transformer layers, skip-connections, and multi-scale pyramid
representation. Compared with the most widely used CNN-based conditional GAN
models (namely pix2pix and pix2pixHD), our model PTNet shows superior
performance in terms of synthesis accuracy and model size. Notably, PTNet does
not require any type of adversarial training and can be easily trained using
the simple mean squared error loss.
