Decision Making Problems with Funnel Structure: A Multi-Task Learning
  Approach with Application to Email Marketing Campaigns
  This paper studies the decision making problem with Funnel Structure. Funnel
structure, a well-known concept in the marketing field, occurs in those systems
where the decision maker interacts with the environment in a layered manner
receiving far fewer observations from deep layers than shallow ones. For
example, in the email marketing campaign application, the layers correspond to
Open, Click and Purchase events. Conversions from Click to Purchase happen very
infrequently because a purchase cannot be made unless the link in an email is
clicked on.
  We formulate this challenging decision making problem as a contextual bandit
with funnel structure and develop a multi-task learning algorithm that
mitigates the lack of sufficient observations from deeper layers. We analyze
both the prediction error and the regret of our algorithms. We verify our
theory on prediction errors through a simple simulation. Experiments on both a
simulated environment and an environment based on real-world data from a major
email marketing company show that our algorithms offer significant improvement
over previous methods.
