Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth
  with RGB Fusion in Challenging Environments
  Indirect Time-of-Flight (I-ToF) imaging is a widespread way of depth
estimation for mobile devices due to its small size and affordable price.
Previous works have mainly focused on quality improvement for I-ToF imaging
especially curing the effect of Multi Path Interference (MPI). These
investigations are typically done in specifically constrained scenarios at
close distance, indoors and under little ambient light. Surprisingly little
work has investigated I-ToF quality improvement in real-life scenarios where
strong ambient light and far distances pose difficulties due to an extreme
amount of induced shot noise and signal sparsity, caused by the attenuation
with limited sensor power and light scattering. In this work, we propose a new
learning based end-to-end depth prediction network which takes noisy raw I-ToF
signals as well as an RGB image and fuses their latent representation based on
a multi step approach involving both implicit and explicit alignment to predict
a high quality long range depth map aligned to the RGB viewpoint. We test our
approach on challenging real-world scenes and show more than 40% RMSE
improvement on the final depth map compared to the baseline approach.
