If you are interested in this, the list of papers for the 2015 International Conference on Learning Representations has been released: <a href="http://www.iclr.cc/doku.php?id=iclr2015:main#submissions" rel="nofollow">http://www.iclr.cc/doku.php?id=iclr2015:main#submissions</a><p>There's some pretty good stuff there. I really liked <i>Crypto-Nets: Neural Networks over Encrypted Data</i>[1]:<p><i>The problem we address is the following: how can a user employ a predictive model that is held by a third party, without compromising private information. For example, a hospital may wish to use a cloud service to predict the readmission risk of a patient. However, due to regulations, the patient's medical files cannot be revealed. The goal is to make an inference using the model, without jeopardizing the accuracy of the prediction or the privacy of the data.</i><p>[1] <a href="http://arxiv.org/abs/1412.6181" rel="nofollow">http://arxiv.org/abs/1412.6181</a>
If you see convolution as basically truncated recurrence this approach ties in very strongly to recent approaches to machine translation using recurrent nets. I guess depth <i>should</i> allow you to find longterm dependencies, but the fact that CNN were designed for images which have strong local structure and much weaker long term structure makes me think RNNs are better for language, where we see a lot of important long term dependencies. As an example: "The man with the long brown hair entered the saloon" - I would tie saloon and man as the key pieces of that sentence, but that dependency is pretty long and somewhat different than natural images where you don't really expect the corners of images to have any strong relationship in general.
Would like to actually see these CNN's first hand but if I'm not missing something on a first quick pass, the amazing results of 2013-2014 are continuing. Major gating barriers that have stymied AGI's are disappearing. This result is cool on its own. But, what really excites is how this provides a key foundational building block for even cooler ideas. When you have sentence structure, it naturally lends itself to many unsupervised NLP and learning tasks. Just wow!
How do you guys understand this stuff!?
The topic absolutely fascinated me, but after reading the paper... I just feel dumb. Are there any good resources I could use to better understand machine learning papers such as this? I mean I can't even comprehend the implications this paper could have? Is anyone opening to doing some mentoring?
Gbachik@gmail.com
Why don't we reallocate some of this effort to teaching humans how to use language more effectively? Or is it better to just delegate the responsibility for truth to a seemingly "pure" logic that we like to think exists outside of our human condition?
The progress here is getting spooky, including the success of such networks for captioning arbitrary images and translating between natural languages. I read that one researcher predicts live narration of video within 5 years.<p>A cold-splash-in-the-face intro for laypeople can be found in the TEDx talk of Jeremy Howard (founder of Kaggle):<p><i>The wonderful and terrifying implications of computers that can learn</i> – <a href="https://www.youtube.com/watch?v=xx310zM3tLs" rel="nofollow">https://www.youtube.com/watch?v=xx310zM3tLs</a><p>For cutting-edge research, it seems the "NIPS" conference each December is where many of the new results appear:<p><a href="http://nips.cc/" rel="nofollow">http://nips.cc/</a>
I see Thomas Mikolov, the creator of Google's Word2Vec (<a href="https://www.kaggle.com/c/word2vec-nlp-tutorial/forums/t/12349/word2vec-is-based-on-an-approach-from-lawrence-berkeley-national-lab" rel="nofollow">https://www.kaggle.com/c/word2vec-nlp-tutorial/forums/t/1234...</a>) is referenced in the paper.