Yann gave a talk at my university last week. It's really astounding how far deep learning has come from performing classification and regression to now doing things like reasoning and generation. Does anyone else find this wild? These are tasks that historically required a tremendous amount of domain-specific engineering to perform well. And recurrent memory networks just power right on through them.<p>I understand that learned, dense vector representations are powerful blocks to operate on, but hot damn the trend of general systems that require less domain-specific engineering is an exciting one that blows my mind.
You know, I see many different big names in the ML field shooting for a personal 'assistant' or something like that. For whatever reason I have always thought what would be preferable would be an advocate. A being that would always have your back and be watching out for your best self and for you in general, preventing you from getting screwed by someone trying to make an information asymmetry play or trying to access your data. Somehow I worry that most big companies aren't going to be trying to create something that puts the user first. I'd love to be wrong.
He was right: For the
progress he
is seeking now, neural
networks stand to play
at most a peripheral role.
Instead, he needs some
new and quite different
ideas.<p>That the networks are <i>neural</i>
might mean that they could
simulate a worm; that he got more
out of them, as, say, non-linear
fitting functions, is cute but
not much progress for the
next challenges he has in mind.
FYI this was submitted a few days ago and no one seemed to care:<p><a href="https://news.ycombinator.com/item?id=10521398" rel="nofollow">https://news.ycombinator.com/item?id=10521398</a>
"Unsupervised learning"?<p>Well, setting aside the jargon, one approach is just a statistical hypothesis test, and a lot is known about that.