The False Prophecy of “AGI”: A Quick TL;DR

This feels like another in a long line of questionably written essays that get posted here. As an example:<p>&gt; Right now, every ML “breakthrough” is going towards making addictive media products more addictive and towards advertising that exerts more control over consumers. So this needs to change.<p>&gt; Luckily, regulation is doable! As is fighting the system. I mean, it’s won’t be easy, but local organizations everywhere, such as social democratic organizations, are working together to push for positive economic change. And they’re quite social.<p>A claim backed up by zero evidence . We need &quot;more regulation&quot;? To what end? Even the long form article doesn&#x27;t illustrate the point. It says<p>&gt;If we made a pie chart representing the money that is made from deep learning<p>but doesn&#x27;t bother to present neither a pie chart nor any underlying facts to back up this claim. It also talks about regulating political donations which seems... difficult to connect to the topic at hand.<p>I am in favour of people trying to improve their writing and publishing blog posts but this just looks like the author didn&#x27;t spend much time collecting their thoughts. If they have edited this down, they could have fooled me.<p>I don&#x27;t mean to discourage the author - I am confident they can improve their writing and should definitely attempt to do so. I am just trying to understand why these articles are being upvoted when to my eyes they aren&#x27;t of high quality.

I don&#x27;t have much to say about many of the points, and I don&#x27;t believe the conclusion about regulation follows, but<p>&gt; Any “AGI” research that compares machine “performance” to humans contains a notable fallacy, the “task fallacy”<p>This is bang on. Everyone doesn&#x27;t do this, but I have seen a subtle migration from AGI being roughly defined as a human mind in a computer, towards &quot;AI that is as good as or better than a human at something&quot;. The latter is of course nothing like intelligence (a few logic gates are way better than any of us at adding). While it certainly could have societal applications (in the same way an automated loom is better than me at weaving, or a wheel is better than me at moving forward under load efficiently), it doesn&#x27;t displace human intelligence.

I prefer Gwern&#x27;s critique of &quot;Tool AI&quot; as a path toward AGI, similar to her 5th point: <a href="https:&#x2F;&#x2F;www.gwern.net&#x2F;Tool-AI" rel="nofollow">https:&#x2F;&#x2F;www.gwern.net&#x2F;Tool-AI</a><p>The recent interview with John Carmack on Lex Friedman also made me hopeful that at least people in the field (of AGI research) are working of the issues with the current approach of strapping models together directly on the required input, instead of agents in a virtual world (or worse yet, with some connection to the outside world) that&#x27;s running as fast as possible.<p>Gato&#x27;s [1] more general approach of using different inputs and actions in the same format is probably a good path towards this kind of agent-in-a-world AI, which is much closer to how a natural organism interacts and lives: serially, always-on, and in a universe.<p>[1] <a href="https:&#x2F;&#x2F;www.deepmind.com&#x2F;publications&#x2F;a-generalist-agent" rel="nofollow">https:&#x2F;&#x2F;www.deepmind.com&#x2F;publications&#x2F;a-generalist-agent</a>

I don&#x27;t know squat about AI research, but I was excited to hear that Carmack is getting into it in a big way. Even took OPM.<p>This is exciting to me because, for one, he is smart and talented and driven and might be able to make significant advances. But more importantly, it means he has done the initial research of the state of the art, gone through the larval stage as he calls it, and has decided with reasonable confidence that there is much promise for further progress.

AGI is like pronouncements of water or life on [Moon, Mars, x...] lots of similar pronouncements that slowly alter in character over time.<p>After listening to Duncan Russell&#x27;s interview of Blake Lemoine[1], my estimated odds that Google does indeed have something close enough to AGI in house at more than 50%. There are a lot of highly non-standard views of the world on both sides of the conversation, so it&#x27;s a wonderfully chaotic interview, and you might reach far different conclusions.<p>The thing is, that AI is a hive mind, not something you could just section off and use for a single task. It doesn&#x27;t fit the model of AI that commercial organizations need. Utilizing it to do simple tasks would be a morally hazardous thing to do, in my opinion.<p><pre><code> 1 - https:&#x2F;&#x2F;www.duncantrussell.com&#x2F;episodes&#x2F;2022&#x2F;7&#x2F;1&#x2F;blake-lemoine</code></pre>

A bunch of straw man fallacies and nothing new. I don&#x27;t know why I expected more after reading the title.

&gt; There are of course, great use-cases of ML<p>It would be great to hear about them