Three-dimensional virtual refocusing of fluorescence microscopy images
  using deep learning
  Three-dimensional (3D) fluorescence microscopy in general requires axial
scanning to capture images of a sample at different planes. Here we demonstrate
that a deep convolutional neural network can be trained to virtually refocus a
2D fluorescence image onto user-defined 3D surfaces within the sample volume.
With this data-driven computational microscopy framework, we imaged the neuron
activity of a Caenorhabditis elegans worm in 3D using a time-sequence of
fluorescence images acquired at a single focal plane, digitally increasing the
depth-of-field of the microscope by 20-fold without any axial scanning,
additional hardware, or a trade-off of imaging resolution or speed.
Furthermore, we demonstrate that this learning-based approach can correct for
sample drift, tilt, and other image aberrations, all digitally performed after
the acquisition of a single fluorescence image. This unique framework also
cross-connects different imaging modalities to each other, enabling 3D
refocusing of a single wide-field fluorescence image to match confocal
microscopy images acquired at different sample planes. This deep learning-based
3D image refocusing method might be transformative for imaging and tracking of
3D biological samples, especially over extended periods of time, mitigating
photo-toxicity, sample drift, aberration and defocusing related challenges
associated with standard 3D fluorescence microscopy techniques.
