TrimTuner: Efficient Optimization of Machine Learning Jobs in the Cloud
  via Sub-Sampling
  This work introduces TrimTuner, the first system for optimizing machine
learning jobs in the cloud to exploit sub-sampling techniques to reduce the
cost of the optimization process while keeping into account user-specified
constraints. TrimTuner jointly optimizes the cloud and application-specific
parameters and, unlike state of the art works for cloud optimization, eschews
the need to train the model with the full training set every time a new
configuration is sampled. Indeed, by leveraging sub-sampling techniques and
data-sets that are up to 60x smaller than the original one, we show that
TrimTuner can reduce the cost of the optimization process by up to 50x.
Further, TrimTuner speeds-up the recommendation process by 65x with respect to
state of the art techniques for hyper-parameter optimization that use
sub-sampling techniques. The reasons for this improvement are twofold: i) a
novel domain specific heuristic that reduces the number of configurations for
which the acquisition function has to be evaluated; ii) the adoption of an
ensemble of decision trees that enables boosting the speed of the
recommendation process by one additional order of magnitude.
