Related. Others?<p><i>Debunking Statistical AI – Noam Chomsky, Gary Marcus, Jeremy Kahn [video]</i> - <a href="https://news.ycombinator.com/item?id=33857543" rel="nofollow">https://news.ycombinator.com/item?id=33857543</a> - Dec 2022 (19 comments)<p><i>Noam Chomsky: Where Artificial Intelligence Went Wrong (2012)</i> - <a href="https://news.ycombinator.com/item?id=30937760" rel="nofollow">https://news.ycombinator.com/item?id=30937760</a> - April 2022 (2 comments)<p><i>On Chomsky and the Two Cultures of Statistical Learning (2011)</i> - <a href="https://news.ycombinator.com/item?id=16489828" rel="nofollow">https://news.ycombinator.com/item?id=16489828</a> - March 2018 (12 comments)<p><i>On Chomsky and the Two Cultures of Statistical Learning (2011)</i> - <a href="https://news.ycombinator.com/item?id=11951444" rel="nofollow">https://news.ycombinator.com/item?id=11951444</a> - June 2016 (102 comments)<p><i>Norvig vs. Chomsky and the Fight for the Future of AI</i> - <a href="https://news.ycombinator.com/item?id=5318292" rel="nofollow">https://news.ycombinator.com/item?id=5318292</a> - March 2013 (2 comments)<p><i>Noam Chomsky on Where Artificial Intelligence Went Wrong</i> - <a href="https://news.ycombinator.com/item?id=4729068" rel="nofollow">https://news.ycombinator.com/item?id=4729068</a> - Nov 2012 (177 comments)<p><i>Norvig vs. Chomsky and the Fight for the Future of AI</i> - <a href="https://news.ycombinator.com/item?id=4290604" rel="nofollow">https://news.ycombinator.com/item?id=4290604</a> - July 2012 (147 comments)<p><i>Norvig vs. Chomsky and the Fight for the Future of AI</i> - <a href="https://news.ycombinator.com/item?id=2710733" rel="nofollow">https://news.ycombinator.com/item?id=2710733</a> - June 2011 (4 comments)<p><i>On Chomsky and the Two Cultures of Statistical Learning</i> - <a href="https://news.ycombinator.com/item?id=2591154" rel="nofollow">https://news.ycombinator.com/item?id=2591154</a> - May 2011 (107 comments)
The debate is mostly about:<p>Are opaque probabilistic models scientific?<p>David Mumford's stance:<p>"This paper is a meant to be a polemic which argues for a very fundamental
point: that stochastic models and statistical reasoning are more relevant i) to
the world, ii) to science and many parts of mathematics and iii) particularly
to understanding the computations in our own minds, than exact models and
logical reasoning"<p><a href="https://www.dam.brown.edu/people/mumford/beyond/papers/2000b--DawningAgeStoch-NC.pdf" rel="nofollow">https://www.dam.brown.edu/people/mumford/beyond/papers/2000b...</a>
I don't know but I hope that Chomsky is less wrong. Because if statistical methods reach an asymptote, we will have no choice but to try to better understand the principles and foundations.<p>If statistical methods do not reach an asymptote, I don't think we will have the incentive to reach a deeper understanding
I think the thing to note about today's large language models is that they aren't purely statistical. The topology of the neural networks behind them has been explicitly defined by the creators of those systems. They are not 'tabula rasa' as some might suppose them to be.
I think it should be marked as being from 2017. Also, I don't see much point in this article. It just butchers Norvig's article into a bunch of quotes even though his article is quite accessible and not very long.
Chomsky has already addressed Norvig's objections before he even made them as anyone how has actually listened to what Chomsky has said would know.<p>> Norvig's reply: He agrees, but engineering success often facilitates scientific success.<p>There has been virtually no progress in understanding of the human language faculty from probabilistic models or LLMs.<p>> Norvig's reply: Science is both description and explanation; you can't have one without the other; in the history of science, the laborious accumulation of data is the usual mode of operation.<p>Probabilistic models don't describe anything in a way that leads to understanding (as the fact that no progress in understanding has been made shows).<p>> people actually generate and understand language in some rich statistical sense (maybe with statistical models several layers deep, like the modern AI models of speech recognition).<p>They do not; there are studies which Chomsky cites involving trying to learning "impossible" non-structural languages that give strong evidence that this is not the case.<p>> Norvig's reply: Certain advances in statistical learning methods provide reason to believe that such learning methods will be able to do the job.<p>It has nothing to do with the human language faculty.<p>> My conclusion is that 100% of these articles and awards are more about "accurately modeling the world" than they are about "providing insight," although they all have some theoretical insight component as well.<p>If you have a black box machine and you write a paper that says the black box reproduces some natural phenomenon with a billion of its knobs to turned to these specific settings you have wasted everyone's time.<p>> Norvig illustrates that rules about language do not always capture the right phenomena. (i before e)<p>The fundamental character of human language has nothing to do with spelling.<p>> [Norvig]: so any valid criticism of probabilistic models would have to be because they are too expressive, not because they are not expressive enough.<p>Yes Chomsky has explicitly said this. Any model that accepts virtually everything is a bad model.<p>I don't have time to go through the rest.
What’s really interesting is that entire Chomskian syntax worldview is fairly pseudo-scientific in nature. Most of these papers are about taking an essentially Turing-complete computation system and tweaking it until it can solve a specific riddle. Rinse and repeat. Most of the arguments (like the poverty of stimulus) are purely authoritarian as well.
This is cool, especially because its already 6 years old and I think not much has changed. Can anyone here speak to the current SOTA of explaining whats going on inside a neural net?<p>If we go: problem -> nnet solution -> explanation of nnet -> insight into problem that would still be very significant to the scientific process.
So, if the people rise up in rebellion against a believe system, like sovjet-socialism and the defenders of that believe system thus decide to decline those people - or even regions agency over there own destiny. Does that make those defenders of the believe system pseudo-scientific, because they fail to update there heuristic models?
The weirdest thing to me is that a person with a high personal, financial involvement in the subject went and took that quote from an old man (which to your and my understanding only states that these things are not linguistic "science", but they solve problems alright), then created a strawman from thin air (points A-E) to then say "oh, all my arguments are void, statistical models are great, don't you dare criticizing me, you old fool".<p>And then he went and took "Science", aka the epitome of publish or perish academia and tried to argue that all this looks the same as the thing he does. Oh well, who would have guessed...<p>The looks on this are weird, even more so as GPT nowadays works wonders, but still doesn't help explaining why and how language evolved (which seems to be the idea of linguistics, no?).