Just yesterday there was a big discussion on here about how academic papers needlessly complicate simple ideas. Mainly by replacing nice explanations with impenetrable math notation in an attempt to seem more formal. This paper is very guilty of this.<p>E.g. page 5. They attempt to explain a really simple idea, that they generated images of random lines at a random angle. Then labelled the lines positive or negative examples, based on whether the angle was greater than 90 degrees or not. Then they take sets of these examples. And label them based on whether they contain an even or odd number of positive examples.<p>They take several paragraphs over half a page to explain this. Filled with dense mathematical notation. If you don't know what symbols like U, :, ->, or ~ mean, you are screwed because that's not googleable. It takes way longer to parse than it should. Especially since I just wanted to quickly skim the ideas, not painfully reverse engineer them.<p>Hell, even <i>the concept of even or odd</i>, is pointlessly redefined and complicated as multiplying + or - 1's together. I was scratching my head for a few minutes just trying to figure out what the purpose of that was. It's like reading bad code without any comments. Even if you are very familiar with the language and know what the code does, it takes a lot of effort to figure out <i>why</i> it's that way. If it's not explained properly.<p>The worst part is, no one ever complains about this stuff because they are afraid of looking stupid. I sure fear that by posting this very comment. I actually am familiar with the notation used in this example. I still find it unnecessary and exhausting to decode.
I think some of the most exciting and interesting work comes out of proving, not just capabilities, but constraints for systems, be it Gödel, Shannon, Aaronson, or any of the others in the smaller-than-desirable tradition of those who say, "No". I think a better understanding what Deep Learning <i>can't</i> do (well) is fertile material for better understanding the kinds of problems it <i>can</i> do, and am very excited to see more work in this space, and movement towards an underlying structural theory.
The lead author will be giving a talk on this work next week (which will be live streamed and recorded) as part of a workshop on Representation Learning:<p><a href="https://simons.berkeley.edu/talks/shai-shalev-shwartz-2017-3-28" rel="nofollow">https://simons.berkeley.edu/talks/shai-shalev-shwartz-2017-3...</a>
Link to the PDF: <a href="https://arxiv.org/pdf/1703.07950.pdf" rel="nofollow">https://arxiv.org/pdf/1703.07950.pdf</a>