Quality meets Diversity: A Model-Agnostic Framework for Computerized
  Adaptive Testing
  Computerized Adaptive Testing (CAT) is emerging as a promising testing
application in many scenarios, such as education, game and recruitment, which
targets at diagnosing the knowledge mastery levels of examinees on required
concepts. It shows the advantage of tailoring a personalized testing procedure
for each examinee, which selects questions step by step, depending on her
performance. While there are many efforts on developing CAT systems, existing
solutions generally follow an inflexible model-specific fashion. That is, they
need to observe a specific cognitive model which can estimate examinee's
knowledge levels and design the selection strategy according to the model
estimation. In this paper, we study a novel model-agnostic CAT problem, where
we aim to propose a flexible framework that can adapt to different cognitive
models. Meanwhile, this work also figures out CAT solution with addressing the
problem of how to generate both high-quality and diverse questions
simultaneously, which can give a comprehensive knowledge diagnosis for each
examinee. Inspired by Active Learning, we propose a novel framework, namely
Model-Agnostic Adaptive Testing (MAAT) for CAT solution, where we design three
sophisticated modules including Quality Module, Diversity Module and Importance
Module. Extensive experimental results on two real-world datasets clearly
demonstrate that our MAAT can support CAT with guaranteeing both quality and
diversity perspectives.
