Diagnostic Assessment Generation via Combinatorial Search
  Initial assessment tests are crucial in capturing learner knowledge states in
a consistent manner. Aside from crafting questions itself, putting together
relevant problems to form a question sheet is also a time-consuming process. In
this work, we present a generic formulation of question assembly and a genetic
algorithm based method that can generate assessment tests from raw
problem-solving history. First, we estimate the learner-question knowledge
matrix (snapshot). Each matrix element stands for the probability that a
learner correctly answers a specific question. We formulate the task as a
combinatorial search over this snapshot. To ensure representative and
discriminative diagnostic tests, questions are selected (1) that has a low root
mean squared error against the whole question pool and (2) high standard
deviation among learner performances. Experimental results show that the
proposed method outperforms greedy and random baseline by a large margin in one
private dataset and four public datasets. We also performed qualitative
analysis on the generated assessment test for 9th graders, which enjoys good
problem scatterness across the whole 9th grader curriculum and decent
difficulty level distribution.
