Automatic Estimation of Ulcerative Colitis Severity from Endoscopy
  Videos using Ordinal Multi-Instance Learning
  Ulcerative colitis (UC) is a chronic inflammatory bowel disease characterized
by relapsing inflammation of the large intestine. The severity of UC is often
represented by the Mayo Endoscopic Subscore (MES) which quantifies mucosal
disease activity from endoscopy videos. In clinical trials, an endoscopy video
is assigned an MES based upon the most severe disease activity observed in the
video. For this reason, severe inflammation spread throughout the colon will
receive the same MES as an otherwise healthy colon with severe inflammation
restricted to a small, localized segment. Therefore, the extent of disease
activity throughout the large intestine, and overall response to treatment, may
not be completely captured by the MES. In this work, we aim to automatically
estimate UC severity for each frame in an endoscopy video to provide a higher
resolution assessment of disease activity throughout the colon. Because
annotating severity at the frame-level is expensive, labor-intensive, and
highly subjective, we propose a novel weakly supervised, ordinal classification
method to estimate frame severity from video MES labels alone. Using clinical
trial data, we first achieved 0.92 and 0.90 AUC for predicting mucosal healing
and remission of UC, respectively. Then, for severity estimation, we
demonstrate that our models achieve substantial Cohen's Kappa agreement with
ground truth MES labels, comparable to the inter-rater agreement of expert
clinicians. These findings indicate that our framework could serve as a
foundation for novel clinical endpoints, based on a more localized scoring
system, to better evaluate UC drug efficacy in clinical trials.
