Optimal Model Averaging of Support Vector Machines in Diverging Model
  Spaces
  Support vector machine (SVM) is a powerful classification method that has
achieved great success in many fields. Since its performance can be seriously
impaired by redundant covariates, model selection techniques are widely used
for SVM with high dimensional covariates. As an alternative to model selection,
significant progress has been made in the area of model averaging in the past
decades. Yet no frequentist model averaging method was considered for SVM. This
work aims to fill the gap and to propose a frequentist model averaging
procedure for SVM which selects the optimal weight by cross validation. Even
when the number of covariates diverges at an exponential rate of the sample
size, we show asymptotic optimality of the proposed method in the sense that
the ratio of its hinge loss to the lowest possible loss converges to one. We
also derive the convergence rate which provides more insights to model
averaging. Compared to model selection methods of SVM which require a tedious
but critical task of tuning parameter selection, the model averaging method
avoids the task and shows promising performances in the empirical studies.
