Two-Stream Deep Feature Modelling for Automated Video Endoscopy Data
  Analysis
  Automating the analysis of imagery of the Gastrointestinal (GI) tract
captured during endoscopy procedures has substantial potential benefits for
patients, as it can provide diagnostic support to medical practitioners and
reduce mistakes via human error. To further the development of such methods, we
propose a two-stream model for endoscopic image analysis. Our model fuses two
streams of deep feature inputs by mapping their inherent relations through a
novel relational network model, to better model symptoms and classify the
image. In contrast to handcrafted feature-based models, our proposed network is
able to learn features automatically and outperforms existing state-of-the-art
methods on two public datasets: KVASIR and Nerthus. Our extensive evaluations
illustrate the importance of having two streams of inputs instead of a single
stream and also demonstrates the merits of the proposed relational network
architecture to combine those streams.
