Event-based Synthetic Aperture Imaging with a Hybrid Network
  Synthetic aperture imaging (SAI) is able to achieve the see through effect by
blurring out the off-focus foreground occlusions and reconstructing the
in-focus occluded targets from multi-view images. However, very dense
occlusions and extreme lighting conditions may bring significant disturbances
to the SAI based on conventional frame-based cameras, leading to performance
degeneration. To address these problems, we propose a novel SAI system based on
the event camera which can produce asynchronous events with extremely low
latency and high dynamic range. Thus, it can eliminate the interference of
dense occlusions by measuring with almost continuous views, and simultaneously
tackle the over/under exposure problems. To reconstruct the occluded targets,
we propose a hybrid encoder-decoder network composed of spiking neural networks
(SNNs) and convolutional neural networks (CNNs). In the hybrid network, the
spatio-temporal information of the collected events is first encoded by SNN
layers, and then transformed to the visual image of the occluded targets by a
style-transfer CNN decoder. Through experiments, the proposed method shows
remarkable performance in dealing with very dense occlusions and extreme
lighting conditions, and high quality visual images can be reconstructed using
pure event data.
