Low-latency job scheduling with preemption for the development of deep
  learning
  One significant challenge in the job scheduling of computing clusters for the
development of deep learning algorithms is the efficient scheduling of
trial-and-error (TE) job, the type of job in which the users seek to conduct
small-scale experiments while monitoring their processes. Unfortunately, the
existing job schedulers to date do not feature well-balanced scheduling for the
mixture of TE jobs and best-effort (BE) jobs, or they can handle the mixture in
limited situations at most. To fill in this niche, we propose an algorithm that
can significantly reduce the latency of TE jobs in versatile situations without
greatly elongating the slowdown of the BE jobs. Our algorithm efficiently
schedules both TE and BE jobs by selectively preempting the BE jobs that can
be, when the time comes, resumed without much delay. In our simulation study
with synthetic and real workloads, we were able to reduce the 95th percentile
of the slowdown rates for the TE jobs in the standard FIFO strategy by 96.6%,
while compromising the median of the BE slowdown rates by only 18.0% and the
95th percentile by only 23.9%.
