Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutation
  In the past decade, Deep Learning (DL) systems have been widely deployed in
various domains to facilitate our daily life. Meanwhile, it is extremely
challenging to ensure the correctness of DL systems (e.g., due to their
intrinsic nondeterminism), and bugs in DL systems can cause serious
consequences and may even threaten human lives. In the literature, researchers
have explored various techniques to test, analyze, and verify DL models, since
their quality directly affects the corresponding system behaviors. Recently,
researchers have also proposed novel techniques for testing the underlying
operator-level DL libraries (such as TensorFlow and PyTorch), which provide
general binary implementations for each high-level DL operator for running
various DL models on many platforms. However, there is still limited work
targeting the reliability of the emerging tensor compilers, which aim to
directly compile high-level tensor computation graphs into high-performance
binaries for better efficiency, portability, and scalability. In this paper, we
target the important problem of tensor compiler testing, and have proposed
Tzer, a practical fuzzing technique for the widely used TVM tensor compiler.
Tzer focuses on mutating the low-level Intermediate Representation (IR) for TVM
due to the limited mutation space for the high-level IR. More specifically,
Tzer leverages both general-purpose and tensor-compiler-specific mutators
guided by coverage feedback for evolutionary IR mutation; furthermore, Tzer
also performs pass mutation in tandem with IR mutation for more effective
fuzzing. Our results show that Tzer substantially outperforms existing fuzzing
techniques on tensor compiler testing, with 75% higher coverage and 50% more
valuable tests than the 2nd-best technique. To date, Tzer has detected 49
previously unknown bugs for TVM, with 37 bugs confirmed and 25 bugs fixed (PR
merged).
