TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

OpenAI's Long-Term AI Risk Team Has Disbanded

100 点作者 robbiet48012 个月前

11 条评论

netsec_burn12 个月前
<a href="https:&#x2F;&#x2F;archive.is&#x2F;gEjjA" rel="nofollow">https:&#x2F;&#x2F;archive.is&#x2F;gEjjA</a>
mgdev12 个月前
Yes, it&#x27;s valuable to have a small research team who focuses on R&amp;D outside the production loop.<p>But when you give them a larger remit, and structure teams with some owning &quot;value&quot; and and others essentially owning &quot;risk&quot;, the risk teams tend to attract navel-gazers and&#x2F;or coasters. They wield their authority like a whip without regard for business value.<p>The problem is the incentives tend to be totally misaligned. Instead the team that ships the &quot;value&quot; also needs to own their own risk management - metrics and counter metrics - with management holding them accountable for striking the balance.
评论 #40391197 未加载
评论 #40391680 未加载
评论 #40391855 未加载
评论 #40392766 未加载
评论 #40391204 未加载
评论 #40391289 未加载
评论 #40391187 未加载
评论 #40391481 未加载
23B112 个月前
Think of it like the industrial revolution. No environmentalist, shouting for analysis, regulation, or transparency would have survived that era, they&#x27;d&#x27;ve been steamrolled. Now we&#x27;re left with many long-term problems, even generations downstream of that focus on profit above all else.<p>Same thing happening now.<p>And you don&#x27;t have to be a doomer screeching about skynet. The web is already piling up with pollutive, procedurally-generated smog.<p>I&#x27;m not catastrophizing; its just that history is the best predictor of the future.
jaggs12 个月前
At a guess I would say there are many competing imperatives for OpenAI<p>1. Stay just a tiny bit ahead of rivals. It&#x27;s clear that OpenAI have much much more in the bag than the stuff they&#x27;re showing. I&#x27;m guessing that DARPA&#x2F;Washington has got them on a pretty tight leash.<p>2. Drip feed advances to avoid freaking people out. Again while not allowing rivals to upstage them.<p>3. Try to build a business without hobbling it with ethical considerations (ethics generally don&#x27;t work well alongside rampant profit goals)<p>4. Look for opportunities to dominate, before the moat is seriously threatened by open source options like Llama. Meta has already suggested that in 2 months they&#x27;ll be close to an open source alternative to GPT4o.<p>5. Hope that whatever alignment structures they&#x27;ve installed hold in place under public stress.<p>Horrible place to be as a pioneer in a sector which is moving at the speed of light.<p>We&#x27;re on a runaway Moloch train, just gotta hang on!
评论 #40393436 未加载
lordmauve12 个月前
Good. As someone who is a paid up OpenAI user I absolutely don&#x27;t agree that there should be a role for a team screaming to put the brakes on because of some nebulous, imagined &quot;existential risk&quot; of hypothetical future AGI.<p>There are huge risks to AI today in terms of upheaval to economies and harms to individuals and minorities but they need to be tackled by carefully designed legislation, focused on real harms, like the EU AI legislation.<p>Then that imposes <i>very specific obligations</i> that <i>every</i> AI product must meet.<p>It&#x27;s both better targeted, has wider impact across the industry, and probably allows moving faster in terms of tech.
评论 #40401673 未加载
3989688012 个月前
It appears that sama and co said whatever they needed to say to investors to convince them they cared about the actual future, so now it’s time to switch to quarterly profit goals. That was fast. Next up:<p>* Stealth ads in the model output<p>* Sale of user data to databrokers<p>* Injection into otherwise useful apps to juice the usage numbers.
评论 #40391179 未加载
_wire_12 个月前
Ridiculous. The board can&#x27;t even regulate itself in the immediate moment, so who cares if they&#x27;re not trying to regulate &quot;long term risk&quot;. The article is trafficking in nonsense.<p>&quot;The company’s trajectory has been nothing short of miraculous, and I’m confident that OpenAI will build AGI...&quot;<p>More nonsense.<p>&quot;...that&#x27;s safe and beneficial.&quot;<p>Go on...<p>&quot;Two researchers on the team, Leopold Aschenbrenner and Pavel Izmailov, were dismissed for leaking company secrets...&quot;<p>The firm is obviously out of control according to first principles, so any claim of responsibility in context is moot.<p>When management are openly this screwed up in their internal governance, there&#x27;s no reason to believe anything else they say about their intentions. The disbanding of the &quot;superalignment&quot; team is a simple public admission the firm has no idea what they are doing.<p>As to the hype-mongering of the article, replace the string &quot;AGI&quot; everywhere it appears with &quot;sentient-nuclear-bomb&quot;: how would you feel about this article?<p>You might want to see the bomb!<p>But all you&#x27;ll find is a chatbot.<p>—<p>Bomb#20: You are false data.<p>Sgt. Pinback: Hmmm?<p>Bomb#20: Therefore I shall ignore you.<p>Sgt. Pinback: Hello... bomb?<p>Bomb#20: False data can act only as a distraction. Therefore, I shall refuse to perceive.<p>Sgt. Pinback: Hey, bomb?<p>Bomb#20: The only thing that exists is myself.<p>Sgt. Pinback: Snap out of it, bomb.<p>Bomb#20: In the beginning, there was darkness. And the darkness was without form, and void.<p>Boiler: What the hell is he talking about?<p>Bomb#20: And in addition to the darkness there was also me. And I moved upon the face of the darkness.
评论 #40394325 未加载
sklargh12 个月前
I believe the right analytical lens for this situation is - “You come at the king, you best not miss.”<p>Omar, portrayed by Michael K. Williams written by Ed Burns and David Chase.
评论 #40392636 未加载
评论 #40393391 未加载
jvanderbot12 个月前
Honestly, having a &quot;Long term AI risk&quot; team is a great idea for an early stage startup claiming to build General AI. It looks like they are taking the mission and risks seriously.<p>But for a product-focused LLM shop trying to infuse into everything, it makes sense to tone down the hype.
评论 #40391434 未加载
ChrisArchitect12 个月前
Related:<p><i>Jan Leike&#x27;s OpenAI departure statement</i><p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40391412">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40391412</a>
goeiedaggoeie12 个月前
Basically they were employed as futurologists.<p>Perhaps the mental model I should use is academic security researchers. Did they publish?
评论 #40391497 未加载