Unified Models of Human Behavioral Agents in Bandits, Contextual Bandits
  and RL
  Artificial behavioral agents are often evaluated based on their consistent
behaviors and performance to take sequential actions in an environment to
maximize some notion of cumulative reward. However, human decision making in
real life usually involves different strategies and behavioral trajectories
that lead to the same empirical outcome. Motivated by clinical literature of a
wide range of neurological and psychiatric disorders, we propose here a more
general and flexible parametric framework for sequential decision making that
involves a two-stream reward processing mechanism. We demonstrated that this
framework is flexible and unified enough to incorporate a family of problems
spanning multi-armed bandits (MAB), contextual bandits (CB) and reinforcement
learning (RL), which decompose the sequential decision making process in
different levels. Inspired by the known reward processing abnormalities of many
mental disorders, our clinically-inspired agents demonstrated interesting
behavioral trajectories and comparable performance on simulated tasks with
particular reward distributions, a real-world dataset capturing human
decision-making in gambling tasks, and the PacMan game across different reward
stationarities in a lifelong learning setting.
