A framework for predicting, interpreting, and improving Learning
  Outcomes
  It has long been recognized that academic success is a result of both
cognitive and non-cognitive dimensions acting together. Consequently, any
intelligent learning platform designed to improve learning outcomes (LOs) must
provide actionable inputs to the learner in these dimensions. However,
operationalizing such inputs in a production setting that is scalable is not
trivial. We develop an Embibe Score Quotient model (ESQ) to predict test scores
based on observed academic, behavioral and test-taking features of a student.
ESQ can be used to predict the future scoring potential of a student as well as
offer personalized learning nudges, both critical to improving LOs. Multiple
machine learning models are evaluated for the prediction task. In order to
provide meaningful feedback to the learner, individualized Shapley feature
attributions for each feature are computed. Prediction intervals are obtained
by applying non-parametric quantile regression, in an attempt to quantify the
uncertainty in the predictions. We apply the above modelling strategy on a
dataset consisting of more than a hundred million learner interactions on the
Embibe learning platform. We observe that the Median Absolute Error between the
observed and predicted scores is 4.58% across several user segments, and the
correlation between predicted and observed responses is 0.93. Game-like what-if
scenarios are played out to see the changes in LOs, on counterfactual examples.
We briefly discuss how a rational agent can then apply an optimal policy to
affect the learning outcomes by treating the above model like an Oracle.
