GenRadar: Self-supervised Probabilistic Camera Synthesis based on Radar
  Frequencies
  Autonomous systems require a continuous and dependable environment perception
for navigation and decision-making, which is best achieved by combining
different sensor types. Radar continues to function robustly in compromised
circumstances in which cameras become impaired, guaranteeing a steady inflow of
information. Yet, camera images provide a more intuitive and readily applicable
impression of the world. This work combines the complementary strengths of both
sensor types in a unique self-learning fusion approach for a probabilistic
scene reconstruction in adverse surrounding conditions. After reducing the
memory requirements of both high-dimensional measurements through a decoupled
stochastic self-supervised compression technique, the proposed algorithm
exploits similarities and establishes correspondences between both domains at
different feature levels during training. Then, at inference time, relying
exclusively on radio frequencies, the model successively predicts camera
constituents in an autoregressive and self-contained process. These discrete
tokens are finally transformed back into an instructive view of the respective
surrounding, allowing to visually perceive potential dangers for important
tasks downstream.
