Synergy: Resource Sensitive DNN Scheduling in Multi-Tenant Clusters
  Training Deep Neural Networks (DNNs) is a widely popular workload in both
enterprises and cloud data centers. Existing schedulers for DNN training
consider GPU as the dominant resource, and allocate other resources such as CPU
and memory proportional to the number of GPUs requested by the job.
Unfortunately, these schedulers do not consider the impact of a job's
sensitivity to allocation of CPU, memory, and storage resources. In this work,
we propose Synergy, a resource-sensitive scheduler for shared GPU clusters.
Synergy infers the sensitivity of DNNs to different resources using optimistic
profiling; some jobs might benefit from more than the GPU-proportional
allocation and some jobs might not be affected by less than GPU-proportional
allocation. Synergy performs such multi-resource workload-aware assignments
across a set of jobs scheduled on shared multi-tenant clusters using a new
near-optimal online algorithm. Our experiments show that workload-aware CPU and
memory allocations can improve average JCT up to 3.4x when compared to
traditional GPU-proportional scheduling.
