Quick Question: Interrupting Users for Microtasks with Reinforcement
  Learning
  Human attention is a scarce resource in modern computing. A multitude of
microtasks vie for user attention to crowdsource information, perform momentary
assessments, personalize services, and execute actions with a single touch. A
lot gets done when these tasks take up the invisible free moments of the day.
However, an interruption at an inappropriate time degrades productivity and
causes annoyance. Prior works have exploited contextual cues and behavioral
data to identify interruptibility for microtasks with much success. With Quick
Question, we explore use of reinforcement learning (RL) to schedule microtasks
while minimizing user annoyance and compare its performance with supervised
learning. We model the problem as a Markov decision process and use Advantage
Actor Critic algorithm to identify interruptible moments based on context and
history of user interactions. In our 5-week, 30-participant study, we compare
the proposed RL algorithm against supervised learning methods. While the mean
number of responses between both methods is commensurate, RL is more effective
at avoiding dismissal of notifications and improves user experience over time.
