Optimal Admission Control for Multiclass Queues with Time-Varying
  Arrival Rates via State Abstraction
  We consider a novel queuing problem where the decision-maker must choose to
accept or reject randomly arriving tasks into a no buffer queue which are
processed by $N$ identical servers. Each task has a price, which is a positive
real number, and a class. Each class of task has a different price distribution
and service rate, and arrives according to an inhomogenous Poisson process. The
objective is to decide which tasks to accept so that the total price of tasks
processed is maximised over a finite horizon. We formulate the problem as a
discrete time Markov Decision Process (MDP) with a hybrid state space. We show
that the optimal value function has a specific structure, which enables us to
solve the hybrid MDP exactly. Moreover, we prove that as the time step is
reduced, the discrete time solution approaches the optimal solution to the
original continuous time problem. To improve the scalability of our approach to
a greater number of task classes, we present an approximation based on state
abstraction. We validate our approach on synthetic data, as well as a real
financial fraud data set, which is the motivating application for this work.
