For context, Rich Sutton is one of the founding fathers of reinforcement learning. He is also known for his 2019 essay, "The Bitter Lesson": <a href="http://incompleteideas.net/IncIdeas/BitterLesson.html" rel="nofollow noreferrer">http://incompleteideas.net/IncIdeas/BitterLesson.html</a>
This is a link to a Twitter announcing a YouTube coming in 1/2 hour. Does / did the YouTube contain the announcement?<p>Now that Twitter has gone subscriber only, we can't see any context around tweets. I think they should be DOA now when posted to HN as they do not allow for follow up or rtfaing.
Wow<p>This is massive, Rich is probably the singular guy that legitimately has a realistic and long term vision for AGI (and has been mostly correct his whole career)<p>Notably he left the US because he did not want to support the Iraq war and established Alberta as THE reinforcement learning hub since then. Also he did just expand the lab so I'm curious how this will be integrated.<p>Carmack + Sutton working on AGI tells me it is 100% going to happen.
Clearly Rich Sutton is a giant in AI for his contributions in RL, but his recent brief talk "AI Succession" (<a href="https://www.youtube.com/watch?v=NgHFMolXs3U">https://www.youtube.com/watch?v=NgHFMolXs3U</a>) made me worry a bit about the sort of perspective he has on what the "good" outcome here looks like. I say this as someone who is generally optimistic about the promise of AGI. I have no love for the machines as a "species", and by Rich's definition here, yeah, I am "specist" in favor of humans. I think we should use technology for our own benefit, and that it's not inevitable that machines "replace" us.<p>I also think his framing of the counterarguments is not charitable. The serious AI-risk arguments do not argue that a super-intelligent AI will necessarily be evil. They only argue that its motivations will be unaligned with ours, that it will be more competent in achieving goals than us, and that this will be bad for humans as a side effect. I think a good comparison is humans building a highway that incidentally crushes an ant colony. They didn't set out on an evil mission to destroy ants because they hate them, it just happened as a side effect of something the humans wanted. No evil required.
I have studied Rich's RL book (both editions), and enjoy his work and occasional talks on YouTube. Carmack is obviously talented. In the referenced YouTube announcement Carmack basically said that the average person should not really care about what they are doing right now, and the way they are funded they can take their time, not rush out any public systems or projects, etc. - at least that is the way I interpreted what he said.<p>Off topic, but: I used to think AGI was likely, but probably not until maybe 2040. After doing a deep dive into LLMs in the last 2 years, and generally deep learning for the eight years before that, I now think there is some real chance of having something that I consider AGI by 2030. Lately LLMs have become a little useful for graph datastore, KGs, etc. I think the missing piece is the ability of LLMs to handle reasoning, and seeing GPT-4's ability to write Prolog code given a problem description, it doesn't strain my imagination too much to think a breakthrough for reasoning could be here in a year or less.