Anatomy-Aware Self-supervised Fetal MRI Synthesis from Unpaired
  Ultrasound Images
  Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the
developing brain but is not suitable for anomaly screening. For this ultrasound
(US) is employed. While expert sonographers are adept at reading US images, MR
images are much easier for non-experts to interpret. Hence in this paper we
seek to produce images with MRI-like appearance directly from clinical US
images. Our own clinical motivation is to seek a way to communicate US findings
to patients or clinical professionals unfamiliar with US, but in medical image
analysis such a capability is potentially useful, for instance, for US-MRI
registration or fusion. Our model is self-supervised and end-to-end trainable.
Specifically, based on an assumption that the US and MRI data share a similar
anatomical latent space, we first utilise an extractor to determine shared
latent features, which are then used for data synthesis. Since paired data was
unavailable for our study (and rare in practice), we propose to enforce the
distributions to be similar instead of employing pixel-wise constraints, by
adversarial learning in both the image domain and latent space. Furthermore, we
propose an adversarial structural constraint to regularise the anatomical
structures between the two modalities during the synthesis. A cross-modal
attention scheme is proposed to leverage non-local spatial correlations. The
feasibility of the approach to produce realistic looking MR images is
demonstrated quantitatively and with a qualitative evaluation compared to real
fetal MR images.
