GLIB: Towards Automated Test Oracle for Graphically-Rich Applications
  Graphically-rich applications such as games are ubiquitous with attractive
visual effects of Graphical User Interface (GUI) that offers a bridge between
software applications and end-users. However, various types of graphical
glitches may arise from such GUI complexity and have become one of the main
component of software compatibility issues. Our study on bug reports from game
development teams in NetEase Inc. indicates that graphical glitches frequently
occur during the GUI rendering and severely degrade the quality of
graphically-rich applications such as video games. Existing automated testing
techniques for such applications focus mainly on generating various GUI test
sequences and check whether the test sequences can cause crashes. These
techniques require constant human attention to captures non-crashing bugs such
as bugs causing graphical glitches. In this paper, we present the first step in
automating the test oracle for detecting non-crashing bugs in graphically-rich
applications. Specifically, we propose \texttt{GLIB} based on a code-based data
augmentation technique to detect game GUI glitches. We perform an evaluation of
\texttt{GLIB} on 20 real-world game apps (with bug reports available) and the
result shows that \texttt{GLIB} can achieve 100\% precision and 99.5\% recall
in detecting non-crashing bugs such as game GUI glitches. Practical application
of \texttt{GLIB} on another 14 real-world games (without bug reports) further
demonstrates that \texttt{GLIB} can effectively uncover GUI glitches, with 48
of 53 bugs reported by \texttt{GLIB} having been confirmed and fixed so far.
