Programmable 3D snapshot microscopy with Fourier convolutional networks
  3D snapshot microscopy enables fast volumetric imaging by capturing a 3D
volume in a single 2D camera image and performing computational reconstruction.
Fast volumetric imaging has a variety of biological applications such as whole
brain imaging of rapid neural activity in larval zebrafish. The optimal
microscope design for this optical 3D-to-2D encoding is both sample- and
task-dependent, with no general solution known. Deep learning based decoders
can be combined with a differentiable simulation of an optical encoder for
end-to-end optimization of both the deep learning decoder and optical encoder.
This technique has been used to engineer local optical encoders for other
problems such as depth estimation, 3D particle localization, and lensless
photography. However, 3D snapshot microscopy is known to require a highly
non-local optical encoder which existing UNet-based decoders are not able to
engineer. We show that a neural network architecture based on global kernel
Fourier convolutional neural networks can efficiently decode information from
multiple depths in a volume, globally encoded across a 3D snapshot image. We
show in simulation that our proposed networks succeed in engineering and
reconstructing optical encoders for 3D snapshot microscopy where the existing
state-of-the-art UNet architecture fails. We also show that our networks
outperform the state-of-the-art learned reconstruction algorithms for a
computational photography dataset collected on a prototype lensless camera
which also uses a highly non-local optical encoding.
