The OP is including a lot of different concepts under the neural network umbrella. Things like restricted boltzmann machines and hierarchical temporal memory are technically neural networks, but many computer scientists would consider them to be different enough in approach to think of them separately. ie, you wouldn't say "let's use a type of neural network to solve this problem" you would probably say "let's use a restricted boltzmann machine".<p>It is true that these things are becoming more popular. I've found in practice that a modern computer scientist is still more likely to solve a simple learning problem with some form of regression, if only because it's faster than training a NN.