Global Pixel Transformers for Virtual Staining of Microscopy Images
  Visualizing the details of different cellular structures is of great
importance to elucidate cellular functions. However, it is challenging to
obtain high quality images of different structures directly due to complex
cellular environments. Fluorescence staining is a popular technique to label
different structures but has several drawbacks. In particular, label staining
is time consuming and may affect cell morphology, and simultaneous labels are
inherently limited. This raises the need of building computational models to
learn relationships between unlabeled microscopy images and labeled
fluorescence images, and to infer fluorescence labels of other microscopy
images excluding the physical staining process. We propose to develop a novel
deep model for virtual staining of unlabeled microscopy images. We first
propose a novel network layer, known as the global pixel transformer layer,
that fuses global information from inputs effectively. The proposed global
pixel transformer layer can generate outputs with arbitrary dimensions, and can
be employed for all the regular, down-sampling, and up-sampling operators. We
then incorporate our proposed global pixel transformer layers and dense blocks
to build an U-Net like network. We believe such a design can promote feature
reusing between layers. In addition, we propose a multi-scale input strategy to
encourage networks to capture features at different scales. We conduct
evaluations across various fluorescence image prediction tasks to demonstrate
the effectiveness of our approach. Both quantitative and qualitative results
show that our method outperforms the state-of-the-art approach significantly.
It is also shown that our proposed global pixel transformer layer is useful to
improve the fluorescence image prediction results.
