"Deep" is such a good prefix for all sorts of Deep Learning, Deep Math, Deep Thinking, Deep Engineering etc. Wonder if the networks were originally called Thick neural networks, would the ML/AI revolution as we know it still happened?
Is there any good reason why a fully-connected network needs more than one hidden layer? Theoretically, any non-linear function could be mapped into a fqn with only one hidden layer. Does deep has anything to do with fqns or only with cnns?
As soon as I saw the word Deep I stopped continuing.<p>Nnets have always been multi layer since they were invented. That’s the whole idea of progressive feature extraction, and the analogy with biological brain. Theoreticians referred to them properly as nnets or multilayer nnets. Later experimentalists simulated them, thanks to the availability of the computing resources, and experimentally verified that a multi layer nnet can be more efficient than a single layer one. They added superficial terms “deep” and “AI,” “singularity,” etc., which the media and tech industry amplified for obvious reasons.