Is rotation forest the best classifier for problems with continuous
  features?
  In short, our experiments suggest that yes, on average, rotation forest is
better than the most common alternatives when all the attributes are
real-valued. Rotation forest is a tree based ensemble that performs transforms
on subsets of attributes prior to constructing each tree. We present an
empirical comparison of classifiers for problems with only real-valued
features. We evaluate classifiers from three families of algorithms: support
vector machines; tree-based ensembles; and neural networks tuned with a large
grid search. We compare classifiers on unseen data based on the quality of the
decision rule (using classification error) the ability to rank cases (area
under the receiver operating characteristic) and the probability estimates
(using negative log likelihood). We conclude that, in answer to the question
posed in the title, yes, rotation forest is significantly more accurate on
average than competing techniques when compared on three distinct sets of
datasets. Further, we assess the impact of the design features of rotation
forest through an ablative study that transforms random forest into rotation
forest. We identify the major limitation of rotation forest as its scalability,
particularly in number of attributes. To overcome this problem we develop a
model to predict the train time of the algorithm and hence propose a contract
version of rotation forest where a run time cap is imposed {\em a priori}. We
demonstrate that on large problems rotation forest can be made an order of
magnitude faster without significant loss of accuracy. We also show that there
is no real benefit (on average) from tuning rotation forest. We maintain that
without any domain knowledge to indicate an algorithm preference, rotation
forest should be the default algorithm of choice for problems with continuous
attributes.
