Jan Leike's OpenAI departure statement

&quot;Learn to feel the AGI&quot;<p>shades of<p>&gt; Anticipating the arrival of this all-powerful technology, Sutskever began to behave like a spiritual leader, three employees who worked with him told us. His constant, enthusiastic refrain was “feel the AGI,” a reference to the idea that the company was on the cusp of its ultimate goal. At OpenAI’s 2022 holiday party, held at the California Academy of Sciences, Sutskever led employees in a chant: “Feel the AGI! Feel the AGI!” The phrase itself was popular enough that OpenAI employees created a special “Feel the AGI” reaction emoji in Slack.<p>From the Atlantic article. He seems a serious person but it&#x27;s hard to take this circle seriously.

This won&#x27;t make a dent in the logical armor of AI optimists:<p>[ ] If you are not intimately familiar with the development of AI, your warnings on safety can be disregarded due to your basic ignorance about the development of AI<p>[x] If you are intimately familiar with the development of AI, your warnings on safety can be disregarded due to potential conflicts of interest and koolaid drinking<p>Unbridled optimism lives another day!

&quot;sailing against the wind&quot; is a very apt description of hardlining Yuddite philosophy when your company&#x27;s models got <i>maybe</i> 20% better in the past two years (original GPT4 is <i>still</i> the best model I&#x27;ve dealt with to this day), while local models got 1000% better.<p>we should all thank G-d these people weren&#x27;t around during the advent of personal computing and the internet - we&#x27;d have word filters in our fucking text processors and publishing something on the internet would require written permission from your local DEI commissar.<p>arrogance, pure fucking hubris brought about by the incomprehensibly stupid assumption that they will get to be the stewards of this technology.

Thank you Jan for your work and your courage to act and speak candidly about these important topics.<p>Open question to HN: to your knowledge&#x2F;experience which AGI-building companies or projects have a culture most closely aligned with keeping safety, security, privacy, etc. as high a priority as “winning the race” on this new frontier land-grab? I’d love to find and support those teams over the teams that spend more time focused on getting investment and marketshare.

X.com links are currently broken on HN so I posted a screenshot instead<p><a href="https:&#x2F;&#x2F;x.com&#x2F;janleike&#x2F;status&#x2F;1791498187313963308" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;janleike&#x2F;status&#x2F;1791498187313963308</a>

&gt; “Over the past few months […] we were struggling for compute<p>OpenAI literally said they were setting aside 20% of compute to ensure alignment [1] but if you read the fine print, what they said was that they are “dedicating 20% of the compute we’ve secured ‘to date’ to this effort” (emphasis mine). So if their overall compute has increased by 10x then that 20% is suddenly 2%, right? Is OpenAI going to be responsible or is it just a mad race (modelled from the top) to “win” the AI game?<p>[1] <a href="https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;introducing-superalignment&#x2F;?utm_source=tldrai" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;introducing-superalignment&#x2F;?utm_sou...</a>

Seems very arrogant at the end to say humanity is waiting on you. It’s not like Covid—19 where the population was literally looking to scientist&#x2F;gov for an active solution. AGI is something a select set of computer nerds want, but the public isn’t clamoring for. In fact the entire idea of safety&#x2F;alignment&#x2F;cultural shift shows the potential downside on society if it actually were achieved.<p>Meanwhile deepmind with alpha fold etc is showing AI can help with pressing problems of humanity without AGI as the necessary first step.

So... this is relevant if you think AGI superintelligence is a thing, and not so clearly relevant otherwise?

Seems to me the management isn’t quite as high on their own supply (at least privately) and they do not believe anything even remotely resembling “AGI” is possible in the foreseeable future. Which I, and many other people in this field, do agree with.

&gt; &quot;Learn to feel the AGI.&quot;<p>This is flabbergasted statement to me but is probably the necessary attitude to push the AI&#x2F;ML frontier, I guess.<p>I feel old.

It really baffles me that even extremely knowledgeable insiders like this end up buying the AGI hype. LLMs are extremely novel and have lots of groundbreaking applications, but this idea that we are headed directly into fantasy sci-fi A.I. is completely bonkers.<p>It&#x27;s one thing to push this kind of hype to get people talking about A.I. if you&#x27;re trying to capitalize on this space. It&#x27;s something else entirely to swallow your own marketing BS as if it were gospel.<p>But let&#x27;s face it, this guy probably isn&#x27;t serious, he&#x27;s just spewing more hype upon departing OpenAI looking for the next tech company to hire him.

I was and am very skeptical of the &quot;superalignment&quot; plan (which was &quot;build an AI that does alignment research for us, and then ask it what to do&quot;). But it&#x27;s a bad look for OpenAI if they pledged 20% of their compute for this effort and then just didn&#x27;t actually give them the compute.

He&#x27;s convinced that AGI is an eventuality.<p>His call for preparation makes it sound like it&#x27;s near.

The only sane person in AI is LeCun.

It is excellent that he is speaking out.<p>I don&#x27;t believe we can create something more intelligent than us. Smarter, yes, but not more intelligent. We will always have a deeper and more nuanced understanding of reality than our artificial counterparts. But it does not mean that AI can not be a danger.