Comparison of 14 different families of classification algorithms on 115
  binary datasets
  We tested 14 very different classification algorithms (random forest,
gradient boosting machines, SVM - linear, polynomial, and RBF - 1-hidden-layer
neural nets, extreme learning machines, k-nearest neighbors and a bagging of
knn, naive Bayes, learning vector quantization, elastic net logistic
regression, sparse linear discriminant analysis, and a boosting of linear
classifiers) on 115 real life binary datasets. We followed the Demsar analysis
and found that the three best classifiers (random forest, gbm and RBF SVM) are
not significantly different from each other. We also discuss that a change of
less then 0.0112 in the error rate should be considered as an irrelevant
change, and used a Bayesian ANOVA analysis to conclude that with high
probability the differences between these three classifiers is not of practical
consequence. We also verified the execution time of "standard implementations"
of these algorithms and concluded that RBF SVM is the fastest (significantly
so) both in training time and in training plus testing time.
