Neural-based Modeling for Performance Tuning of Spark Data Analytics
  Cloud data analytics has become an integral part of enterprise business
operations for data-driven insight discovery. Performance modeling of cloud
data analytics is crucial for performance tuning and other critical operations
in the cloud. Traditional modeling techniques fail to adapt to the high degree
of diversity in workloads and system behaviors in this domain. In this paper,
we bring recent Deep Learning techniques to bear on the process of automated
performance modeling of cloud data analytics, with a focus on Spark data
analytics as representative workloads. At the core of our work is the notion of
learning workload embeddings (with a set of desired properties) to represent
fundamental computational characteristics of different jobs, which enable
performance prediction when used together with job configurations that control
resource allocation and other system knobs. Our work provides an in-depth study
of different modeling choices that suit our requirements. Results of extensive
experiments reveal the strengths and limitations of different modeling methods,
as well as superior performance of our best performing method over a
state-of-the-art modeling tool for cloud analytics.
