I'm shocked there's so much push back on this here. This was written in 2019 but it applies even more so today. All the recent advances in AI for the past decade have, as a base requirement, massive amounts of compute. Not a proof but it should be a big red flag that if you don't believe Sutton, you should maybe consider.<p>Put it this way. Do you believe premature optimization is the root of all evil? Then why do you believe that some subtle, intricate optimization for machine intelligence will win out over brute compute force? It's not that there's optimizations to be done, it's knowing what optimizations will yield the most value after the space is explored with better computing capabilities.<p>To me, this is almost like a generalized "Proebstings Law" [0], where compiler optimizations give a doubling every 18 years compared, to say, some type of generalized Moore's law which give roughly a doubling in compute every 1.5 to 2 years.<p>[0] <a href="https://zeux.io/2022/01/08/on-proebstings-law/" rel="nofollow">https://zeux.io/2022/01/08/on-proebstings-law/</a>
I think this bitter lesson needs to be taken for a several grains of salt.<p>Number one, the progress in a particular AI field tends to go, at first, from custom to more general algorithms, exactly as Professor Richard Sutton described. However, there is a second part to this progress, where, once we "understood" (which we never really do) the new level of general algorithms (say Transformers in NLP), we begin to put back in the all the things we learned before (say, from Linguistics experience, we put the bias towards compositionality and corresponding tree structures back into the Transformers).<p>Number two, the computationally scalable algorithms always win in the environments where you have unlimited access to the computation and the data, i.e. if you working for Google, Facebook, Alibaba, etc... In other companies, you have limited computational budget and limited data. You could end up putting back-in a lot of sophisticated inductive biases back into your DL algorithms.
Four very interesting concepts in a short space:<p>- "general methods that leverage computation are ultimately the most effective, and by a large margin"<p>- "[search and learning are] methods that continue to scale with increased computation"<p>- "We should stop trying to find simple ways to think about the contents of minds"<p>- "We want AI agents that can discover like we can, not which contain what we have discovered"<p>In other words, computer programs should stop trying be something they are not. They are not AI. Computers are expensive machines that can (very efficiently, and with economies of scale) calculate and present anything that the author of the software desires. It takes actual human intelligence, economics, and ethics to translate that into action.
This is a philosophical and epistemological matter that is often undiscussed. Cognitive philosophers are still hung up on questions from the 70s and their offshoots ("hard problem") etc.<p>On the other hand i am not sure if the "computational" people often know what they are doing. Looking at something like the deep Transformer models, one has to ask if there is any rhyme or reason there, or the thing just works because it's too big and deep. Same even with gradient descent methods, are we sure there aren't closed form solutions instead?<p>There s an even more pessimistic view of this: that the brain and its creations (language, formal systems etc) are resting on the chaos of spiking cells, and are not as ideal as I 'd like them to be.
><i>The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation.</i><p>That doesn't sound right.<p>If there's an "ultimate reason" for computation AI success, is that the problem, of thought, signal processing, pattern matching etc, is not "rational" or based on semantic manipulation, but computational in its nature/substrate in humans too.<p>Computation being increasingly cheaper wouldn't guarantee a success in AI, if those problems weren't inherently solvable by mere throwing computation at them.<p>That is, it's not like we solve it with a brute force over a "proper semantic" approach because brute force got cheaper. It's more like the proper approach is more like our computation one, than a semantic one, and we wrongly assumed that this is not the case in the 70s and 80s, because we only considered the higher levels of conscious/semantic processing in our brains, and not the deep computational processing underneath them.<p>><i>They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess.</i><p>I'd think that most chess playing in humans happens at an uncoscious level, after the player has "trained" their brain, than actual conscious semantic arguing about the next move. The conscious manipulation comes only after huge swaths of moves have been pruned through pattern matching/computation going on in the background.
I see a bunch of comments all basically saying Rick is wrong. It’s unintuitive, I think this is why it feels wrong.<p>Intuition is a curse though. The data to hand clearly indicates that the big (compute), dumb (search and learning) approach is the only way that’s worked. Emergent properties aren’t very satisfying, particularly when we can’t model or understand, but This is The Way.
<a href="https://en.wikipedia.org/wiki/The_Lifecycle_of_Software_Objects" rel="nofollow">https://en.wikipedia.org/wiki/The_Lifecycle_of_Software_Obje...</a><p>Fiction, but closer to reality now than it was when it was written.
Natural selection is a massive amount of computation. It's not like we got to skip that. Any learning that we do without computation is based on structures that were computed by evolution. One way or another the "learning" process is happening somewhere.
"We want AI agents that can discover like we can, not which contain what we have discovered."<p>I wonder if we can even go beyond that and find ways to abstract "discoverability" so that AI agents can evolve into diverse species, perhaps entirely different from our specific ways of discovering. As an analogue, I think of the nervous system of octopuses, which is very different from ours, but capable of amazing feats.
That makes perfect sense.<p>At the same time, will we ever get General AI just by throwing more data into the models?<p>Our biological brains definitely have way more data than the models we have right now but they also have different mechanisms which we barely understand.<p>I'm not confident we'll get human-like behaviour just by throwing infinite amount of data into a model, I think we will need significant changes to how we do NN as well.<p>Or maybe we just grow a brain in the lab, wire some inputs and outputs, train it for 20 years and then treat is a CPU.
On the whole I agree, but having read "Reinforcement Learning: An Introduction" by him and Barto, this article comes across as a hardly nuanced self-endorsement of RL as "the inevitable future of AI". Without mentioning RL by name (but hinting at it with HMM, search&learning- like exploration&exploitation), I think he might be suggesting that any supervised learning is still too specific. Not to mention that he works for DeepMind, which has found fruitful applications of RL.
> [the critics of brute force] said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess.<p>No expert in AI or chess, but I assume that this observation is wrong. Chess players do indeed construct moves based on a search of an interior mental library... a sort of pattern recognition. Any chess players here agree/disagree?<p>As an artist who is amazed at some of the AI art coming out, I can tell you that this is how painters make their paintings. AI is just emulating this. Of course there are differences: AI is not aware of culture, society, new technologies etc. etc. It is also in a closed loop of reference.<p>My colleagues and I are now envisioning a future where AI image rule the roost, but is calling upon the same 'bucket' of human-made existing art. Eventually AI will start using other AI art as reference, and will dissolve into a sort of 'Lorem ipsum' state: an impression of sense without sense. Indeed, it might be argued that to a degree this has already happened.