HN'ers may have a good laugh at these taken from Yan LeCun's page. LeCun made it a tradition to have Hinton jokes in the lines of Chuck Norris ones (or more appropriately Doug McCllroy ones).<p>A few will recall that neural networks all but died from the US after Minsky's damning book. Hinton gave backpropagation which is one of the foundational pillars of feed-forward neural net algorithms. With his new thrust on whats called "deep belief networks" he is challenging his own early seminal contribution in the field. Not often do you see researchers throw away such huge swathes of their own work and start again to solve the same problem. Unless you are Niklaus Wirth of course.<p>Some background is necessary to get the inside jokes, but I have tried to minimize the requirement.<p><pre><code> Geoff Hinton doesn't need to make hidden units. They hide
by themselves when he approaches.
Geoff Hinton discovered how the brain really works.
Once a year for the last 25 years.
Markov random fields think Geoff Hinton is intractable.
Geoff Hinton can make you regret without bounds.
Geoff Hinton doesn't need support vectors. He can
support high-dimensional hyperplanes with his pinky.
All kernels that ever dared approaching Geoff Hinton woke up convolved.
The only kernel Geoff Hinton has ever used is a kernel of truth.
After an encounter with Geoff Hinton, support vectors become unhinged
Geoff Hinton's generalizations are boundless.
Geoff Hinton goes directly to third Bayes.
</code></pre>
Links:
<a href="http://yann.lecun.com/ex/fun/index.html" rel="nofollow">http://yann.lecun.com/ex/fun/index.html</a><p><a href="http://en.wikipedia.org/wiki/Backpropagation" rel="nofollow">http://en.wikipedia.org/wiki/Backpropagation</a><p><a href="http://en.wikipedia.org/wiki/Perceptrons_%28book%29" rel="nofollow">http://en.wikipedia.org/wiki/Perceptrons_%28book%29</a><p><a href="http://en.wikipedia.org/wiki/Niklaus_Wirth" rel="nofollow">http://en.wikipedia.org/wiki/Niklaus_Wirth</a><p>Re: Downvotes. I seem to have touched a nerve. Yes I agree humor is frowned upon here, and I largely agree with that, but when its not humor for humors own sake, I sometimes make an exception. Not everyone would know who Hinton is, but may gauge that he is someone important from these fun anecdotes. But of course you are free to like it or dislike it, no hard feelings either which way.
Sigh. I don't know whether I'm alone in being saddened that many of the excellent thinkers of our time, who once might have spent their lives probing the the very limits of our comprehension in their field, now end up comfortably turning their minds to the services of (in the long term) rather short sighted commercial ends.<p>I can imagine that this sentiment is probably against the prevailing HN mood, but I've been thinking a lot lately about how certain kinds of thought and investigation are only enabled and supported by certain structures. The point of a university used to be being the highest pinnacle of thought. A place where people could have everything not related to intellectual pursuit taken care of so they could devote themselves to the pursuit of greater knowledge. Now that seems to be replaced by a corporate campus.<p>It's a cliche, of course, but how many breakthroughs of meaning are we missing out on because the brightest and best of our generation are now no longer seeking after truth, but instead seeking after a way to make more people click on an ad? Ah well, it's late, and as ever I'm post-sober. Best of luck to him and his team - I've enjoyed his lectures and papers. I hope they may continue, but I sadly suspect not.
Something really interesting must be happening with AI at Google, because in the past few months both Ray Kurzweil (the best-known proponent of the singularity) and Geoff Hinton (the crazy-talented individual who invented deep-belief networks using interconnected "restricted Boltzman machines"[1]) have joined the company.<p>--<p>[1] For an overview of deep belief networks, see these videos: <a href="http://www.youtube.com/watch?v=AyzOUbkUf3M" rel="nofollow">http://www.youtube.com/watch?v=AyzOUbkUf3M</a> , <a href="http://www.youtube.com/watch?v=VdIURAu1-aU" rel="nofollow">http://www.youtube.com/watch?v=VdIURAu1-aU</a> , and <a href="http://www.youtube.com/watch?v=DleXA5ADG78" rel="nofollow">http://www.youtube.com/watch?v=DleXA5ADG78</a>
A related news is "Google acquires Canadian neural networks startup DNNresearch, aims to improve image and voice search"<p><a href="http://thenextweb.com/google/2013/03/12/google-acquires-canadian-neural-networks-startup-dnnresearch-aims-to-improve-image-and-voice-search/" rel="nofollow">http://thenextweb.com/google/2013/03/12/google-acquires-cana...</a>
Google is sucking all the talent and the brain of the world.. i wish at least that the efforts would get published in new papers and open source software..<p>Or the world will enter into a dark age of knowledge, ruled by huge corporations, and everybody else having to share the crumbs that fall of the table..<p>I hope that not all big minds fall of for companies.. or at least that companies start to commit with knowledge shareing to the rest of the world<p>we should stop black box knowledge companies or we are doomed in the long term.. (like the coke secret recipe case)
This is exciting for Hinton I'm sure, but also somewhat worrying for broader community. Hinton has always been very open with his research:<p><a href="http://www.nytimes.com/2012/11/24/science/scientists-see-advances-in-deep-learning-a-part-of-artificial-intelligence.html?pagewanted=2" rel="nofollow">http://www.nytimes.com/2012/11/24/science/scientists-see-adv...</a><p>One of the most striking aspects of the research led by Dr. Hinton is that it has taken place largely without the patent restrictions and bitter infighting over intellectual property that characterize high-technology fields.<p>“We decided early on not to make money out of this, but just to sort of spread it to infect everybody,” he said. “These companies are terribly pleased with this.”
Google is slowly becoming the equivalent of a Silicon Valley think tank. :P<p>Not that it's a bad thing. Lots of smart people did interesting things at IBM, just as people at Microsoft Research are doing some great things - albeit usually without anyone seeing the work in progress.
Poor grad students :-/<p>Anyway, if you are unfamiliar with his work, here's Hinton't Deep Learning Saga ;) for your enjoyment: www.youtube.com/watch?v=mlXzufEk-2E
Slightly off-topic, but I wonder what Dijkstra would feel about the state of CS research if he came to the present time.<p>Hot topics now seem to be statistical ML stuff, and not the deterministic mathematical proofs that he was an advocate of.
In contrast to many who think that people are being shepherded to the search business, I wonder if Google is trying build a new "Bell Labs"-type research division, in which case this is good news.