This may be a very naïve question, but how far are we from toolkit that takes a dataset and a scoring function or goal and figures out which is the best technique, and optimizes it too? Sure, it's probably really resource intensive, but can it be done? Maybe with some kind of genetic algorithm selecting techniques and parameters?<p>Soneone must be working on this.
You can state that a neural network is a hypothesis set, but is it useful when you've got a hypothesis that is 100's of values each with billions of possible values? I mean why talk about that in terms of a set?