Autonomous CRM Control via CLV Approximation with Deep Reinforcement
  Learning in Discrete and Continuous Action Space
  The paper outlines a framework for autonomous control of a CRM (customer
relationship management) system. First, it explores how a modified version of
the widely accepted Recency-Frequency-Monetary Value system of metrics can be
used to define the state space of clients or donors. Second, it describes a
procedure to determine the optimal direct marketing action in discrete and
continuous action space for the given individual, based on his position in the
state space. The procedure involves the use of model-free Q-learning to train a
deep neural network that relates a client's position in the state space to
rewards associated with possible marketing actions. The estimated value
function over the client state space can be interpreted as customer lifetime
value, and thus allows for a quick plug-in estimation of CLV for a given
client. Experimental results are presented, based on KDD Cup 1998 mailing
dataset of donation solicitations.
