This is consistent with our experience running hundreds of Kaggle competitions: for most classification problems, some variation on ensembled decision trees (random forests, gradient boosted machines, etc.) performs the best. This is typically in conjunction with clever data processing, feature selection, and internal validation.<p>One key exception is where the data is richly and hierarchically structured. Text, speech, and visual data falls under this category. In many cases here, variations of neural networks (deep neural nets/CNN's/RNN's/etc.) provide very dramatic improvements.<p>This study does have a couple limitations. The datasets used are very small & form a very biased selection of real-world applications of machine learning. It doesn't consider ensembles of different model types (which I'd expect to provide a consistent but marginal improvement over the results here).
Here's some commentary on the article: <a href="http://www.win-vector.com/blog/2014/12/a-comment-on-preparing-data-for-classifiers/" rel="nofollow">http://www.win-vector.com/blog/2014/12/a-comment-on-preparin...</a>
I think the lesson here has been evident for some time: There is no one best classifier, only classifiers that perform better on particular problems. Random Forests work well because as ensemble methods they can be optimized to explore many different aspects of a data set, but they don't perform significantly better than SVMs or most of the other methods.<p>In practice the solution is often to <i>use a combination of multiple methods</i>. Trees, support vector machines, multilayer perceptrons, gaussian kernels, bagging and boosting. In most applications you don't have choose. Combining the results of all of them together using a weighted average will out perform any of them individually. And in most cases, the whole is greater than the sum of its parts. Each classifier fits a given data set differently and provides its own perspective on the prediction problem. The goal isn't to choose the best one, but to find an ensemble of methods that best explain the patterns and relationships in the data.<p>There are many cases where resource and speed limitations dictate that only one classifier can be tuned and implemented, and in those situations it's good to know which one is 'best'. But when it's possible to build an ensemble out of many different methods it's almost always the best way to go.