Title isn't quite right... The article is estimating the probability of widespread/consumer accessible/cost effective AGI. But AGI can be transformative without being widespread.<p>Replacing all human functions is not nessecary for transformation. A single superhumam AI could cost 1e6 that of a single human, but still do transformative work if it's 1.5x smarter than us.<p>Edit: nevermind... This was an intentional definition of "transformative" made by the authors. Seems wrong to me, but they were up front about it. Not impressed though. Anyone can drum attention by using a non-standard definition of something. But of a cheap trick imo, but that's academia these days I guess... For reference, my definition of transformative would be "responsible for large scale changes to society". In other words, it doesn't need to happen at scale for impact to happen at scale. Replication of a solution is cheap once it is discovered.
Don’t we all known someone who is very widely read, but makes the occasional error or mistake? Gets confused about something, or is asked about a discipline they don’t understand or haven’t researched very deeply? Maybe they misremember a date here or there, but are otherwise fairly intelligent? Maybe they work a data entry job making a middle class salary.<p>This is basically where ChatGPT is at. It’s a very widely read person with an excellent memory and quick mind. It’s probably smarter than the average human (certainly in breadth, often in depth), not to mention a comparison of it to the average human globally.<p>ChatGPT is smarter than the average human already. It doesn’t need agency or a soul to do so. We already have AGI, we just keep moving the goal posts.
So this is some kind of attempt to make a Drake Equation [1] for AGI? That's more useful as a thought experiment than something claimed with scientific precision.<p>[1] <a href="https://en.wikipedia.org/wiki/Drake_equation" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Drake_equation</a>
My opinion on this may be worthless since I have worked in AI since 1982, and I am guilty of enjoying living through several AI hype cycles.<p>After seeing surprising advances in techniques for deep models, then attention+transformer models, I think there is a lot of progress still to be made with cooperating LLMs. I have a simple example of this in the last book I wrote.<p>I have no idea what new ideas will work, but I would be very surprised if every 4 or 5 years new and fresh ideas don’t occur to achieve really good reasoning, counterfactual reasoning, etc. Anyway, putting the chance of transformative AGI below 1% for the next 20 years seems wrong to me.
Let's say we get AGI that costs billions to run, can't do some simple things well, don't have good cheap robots to use, but discovers a way to lengthen the human life significantly and cure most illnesses. Pretty transformative if you ask me.
Using the tool referenced in the paper, my own estimate came to around 25%<p><a href="https://www.tedsanders.com/agi-forecaster/" rel="nofollow noreferrer">https://www.tedsanders.com/agi-forecaster/</a>
“we estimate that transformative AGI by 2043 is 0.4% likely”<p>Not enough significant digits to take seriously. Now if they had said 0.39785% they might have some credibility.
The standard response to this is that the above is commiting the multiple stages fallacy:<p><a href="https://arbital.com/p/multiple_stage_fallacy/" rel="nofollow noreferrer">https://arbital.com/p/multiple_stage_fallacy/</a>
"China has stated plainly it intends to reunify (invade) Taiwan. A majority of its population supports invasion. Its military is preparing to be ready for invasion"<p>Am I reading a paper or listening to Rogan?
Impossible to predict future if future depends on unpredictable factors.<p>I can't shake the feeling that recent developments like LLM's, ChatGPT & co are barking up the wrong tree. Or are missing a key piece of the puzzle. Or that vastly simpler (computationally cheaper) constructs with similar capabilities could be found. That in hindsight (say, 20y from now) we'll say "see, it was really easy!". That missing piece(s) were small but essential. And (perhaps) non-obvious right now.<p>See <a href="https://longbets.org/1/" rel="nofollow noreferrer">https://longbets.org/1/</a><p>Ray Kurzweil's argument is well reasoned & very convincing imho. Computing power will get there or already is. And our understanding of the architecture & function of the human brain is a steadily-completing picture. Not to mention brains of smaller creatures. All it takes is time.<p>Doubtful about timeframe in above bet. And efficiencies of artificial vs. biological brains remains to be seen. But yes, AGI will be achieved. Likely sooner than later.
Previous discussion on LessWrong: <a href="https://www.lesswrong.com/posts/DgzdLzDGsqoRXhCK7/transformative-agi-by-2043-is-less-than-1-likely" rel="nofollow noreferrer">https://www.lesswrong.com/posts/DgzdLzDGsqoRXhCK7/transforma...</a>
and EA Forum: <a href="https://forum.effectivealtruism.org/posts/ARkbWch5RMsj6xP5p/transformative-agi-by-2043-is-less-than-1-likely" rel="nofollow noreferrer">https://forum.effectivealtruism.org/posts/ARkbWch5RMsj6xP5p/...</a><p>Both previous discussions contain multiple independent refutations of the core claim (the argument stacks quite a few errors on top of each other).
This must be one of the stupidest article I have ever read.<p>The first page only shows they have absolutely no clue how statistics works. Of course if you consider every events in the universe completely independent the probability of any large combination of those will be very low. But it is of course not the case with all the conditions they describe, those are highly correlated and P(A|B)P(B)>P(A)P(B) for all of those.