OoDAnalyzer: Interactive Analysis of Out-of-Distribution Samples
  One major cause of performance degradation in predictive models is that the
test samples are not well covered by the training data. Such not
well-represented samples are called OoD samples. In this paper, we propose
OoDAnalyzer, a visual analysis approach for interactively identifying OoD
samples and explaining them in context. Our approach integrates an ensemble OoD
detection method and a grid-based visualization. The detection method is
improved from deep ensembles by combining more features with algorithms in the
same family. To better analyze and understand the OoD samples in context, we
have developed a novel kNN-based grid layout algorithm motivated by Hall's
theorem. The algorithm approximates the optimal layout and has $O(kN^2)$ time
complexity, faster than the grid layout algorithm with overall best performance
but $O(N^3)$ time complexity. Quantitative evaluation and case studies were
performed on several datasets to demonstrate the effectiveness and usefulness
of OoDAnalyzer.
