Image-to-Image MLP-mixer for Image Reconstruction
  Neural networks are highly effective tools for image reconstruction problems
such as denoising and compressive sensing. To date, neural networks for image
reconstruction are almost exclusively convolutional. The most popular
architecture is the U-Net, a convolutional network with a multi-resolution
architecture. In this work, we show that a simple network based on the
multi-layer perceptron (MLP)-mixer enables state-of-the art image
reconstruction performance without convolutions and without a multi-resolution
architecture, provided that the training set and the size of the network are
moderately large. Similar to the original MLP-mixer, the image-to-image
MLP-mixer is based exclusively on MLPs operating on linearly-transformed image
patches. Contrary to the original MLP-mixer, we incorporate structure by
retaining the relative positions of the image patches. This imposes an
inductive bias towards natural images which enables the image-to-image
MLP-mixer to learn to denoise images based on fewer examples than the original
MLP-mixer. Moreover, the image-to-image MLP-mixer requires fewer parameters to
achieve the same denoising performance than the U-Net and its parameters scale
linearly in the image resolution instead of quadratically as for the original
MLP-mixer. If trained on a moderate amount of examples for denoising, the
image-to-image MLP-mixer outperforms the U-Net by a slight margin. It also
outperforms the vision transformer tailored for image reconstruction and
classical un-trained methods such as BM3D, making it a very effective tool for
image reconstruction problems.
