Personalized Execution Time Optimization for the Scheduled Jobs
  Scheduled batch jobs have been widely used on the asynchronous computing
platforms to execute various enterprise applications, including the scheduled
notifications and the candidate computation for the modern recommender systems.
It is important to deliver or update the information to the users at the right
time to maintain the user experience and the execution impact. However, it is
challenging to provide a versatile execution time optimization solution for the
user-basis scheduled jobs to satisfy various product scenarios while
maintaining reasonable infrastructure resource consumption. In this paper, we
describe how we apply a pointwise learning-to-rank approach plus a "best time
policy" in the best time selection. In addition, we propose a value model
approach to efficiently leverage multiple streams of user activity signals in
our scheduling decisions of the execution time. Our optimization approach has
been successfully tested with production traffic that serves billions of users
per day, with statistically significant improvements in various product
metrics, including the notifications and content candidate generation. To the
best of our knowledge, our study represents the first ML-based multi-tenant
solution to the execution time optimization problem for the scheduled jobs at a
large industrial scale.
