Scheduling Jobs with Stochastic Holding Costs
  This paper proposes a learning and scheduling algorithm to minimize the
expected cumulative holding cost incurred by jobs, where statistical parameters
defining their individual holding costs are unknown a priori. In each time
slot, the server can process a job while receiving the realized random holding
costs of the jobs remaining in the system. Our algorithm is a learning-based
variant of the $c\mu$ rule for scheduling: it starts with a preemption period
of fixed length which serves as a learning phase, and after accumulating enough
data about individual jobs, it switches to nonpreemptive scheduling mode. The
algorithm is designed to handle instances with large or small gaps in jobs'
parameters and achieves near-optimal performance guarantees. The performance of
our algorithm is captured by its regret, where the benchmark is the minimum
possible cost attained when the statistical parameters of jobs are fully known.
We prove upper bounds on the regret of our algorithm, and we derive a regret
lower bound that is almost matching the proposed upper bounds. Our numerical
results demonstrate the effectiveness of our algorithm and show that our
theoretical regret analysis is nearly tight.
