Deep Sequential Mosaicking of Fetoscopic Videos
  Twin-to-twin transfusion syndrome treatment requires fetoscopic laser
photocoagulation of placental vascular anastomoses to regulate blood flow to
both fetuses. Limited field-of-view (FoV) and low visual quality during
fetoscopy make it challenging to identify all vascular connections. Mosaicking
can align multiple overlapping images to generate an image with increased FoV,
however, existing techniques apply poorly to fetoscopy due to the low visual
quality, texture paucity, and hence fail in longer sequences due to the drift
accumulated over time. Deep learning techniques can facilitate in overcoming
these challenges. Therefore, we present a new generalized Deep Sequential
Mosaicking (DSM) framework for fetoscopic videos captured from different
settings such as simulation, phantom, and real environments. DSM extends an
existing deep image-based homography model to sequential data by proposing
controlled data augmentation and outlier rejection methods. Unlike existing
methods, DSM can handle visual variations due to specular highlights and
reflection across adjacent frames, hence reducing the accumulated drift. We
perform experimental validation and comparison using 5 diverse fetoscopic
videos to demonstrate the robustness of our framework.
