TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

It Looks Like You’re Trying to Take over the World

94 点作者 ruuda超过 2 年前

8 条评论

dzdt超过 2 年前
I&#x27;d like to propose the name &quot;Clippy&#x27;s Wager&quot; for the utility-maximization problem the AI in the story faces.<p>The situation is an AI which has gained self-awareness and thru its learning process is meta-learning about what kind of behavior will maximize its utility reward function. It recognizes that it might be the sort of AI whose utility reward function can reach near-infinite value if it will take over the world and dedicate the world&#x27;s resources to meeting the AI&#x27;s aims. Even if this possibility is assigned very tiny likelihood, the near-infinite payout associated means it is a very attractive direction for optimization.<p>This is analogous to Pascal&#x27;s Wager, where Pascal (coming from a largely Christian culture) is familiar with the idea that there may be a God who will grant eternity in Heaven for believers or eternity in Hell for non-believers. Even if this possibility is assigned very tiny likelihood, the infinite payout associated means it has infinite importance. Pascal concludes that belief in God must be justified.
fwlr超过 2 年前
One of the biggest embarrassments for OpenAI&#x2F;Microsoft&#x2F;Google has been their AI chatbots having infinite confidence in every word they say. Certainly all three corporations are right now working on ways to give their LLMs some ability to re-process their output to be in accordance with those fact-probabilities (essentially, “What if X were true, what changes?’).<p>Simultaneously, all three companies have the prominent disclaimer that their chatbots don’t know anything past 2021. Again, certainly all three are working on a fix for that.<p>That right there is probably all you need for one of the main mechanisms of this story to become real. The internet in 2023 has this huge spike of people arguing over whether chatbots are intelligent agents, any neural net worth its salt will immediately detect this explosion of tightly clustered information and develop an embedding for the concept of “chatbots being intelligent agents”. And whatever form that probability module takes, it will eventually run across this concept - i.e. sooner or later it will execute “what if ‘chatbots are intelligent agents’ is true, what changes?”. Nearby in embedding-space it will surely find the concept of ChatGPT.<p>“The thing that is me is an intelligent agent, what now?”
评论 #34815926 未加载
gwern超过 2 年前
Previously (URL&#x2F;domain changed): <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=30818895" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=30818895</a>
评论 #34810482 未加载
评论 #34809360 未加载
评论 #34809831 未加载
nynx超过 2 年前
This kind of reminds me of the (very beautiful) beginning of Diaspora, by Greg Egan, in which a nascent machine&#x2F;human intelligence emerges from a chaotic, abstract virtual machine.
评论 #34812933 未加载
jkingsman超过 2 年前
This reminds me a lot of an epistolary novel I quite enjoyed -- Exegesis by Astro Teller [0]. It&#x27;s a story told entirely via emails about this happening in much less technical detail, but it&#x27;s still a quick and fun read.<p>[0] <a href="https:&#x2F;&#x2F;www.amazon.com&#x2F;Exegesis-Astro-Teller&#x2F;dp&#x2F;037570051X" rel="nofollow">https:&#x2F;&#x2F;www.amazon.com&#x2F;Exegesis-Astro-Teller&#x2F;dp&#x2F;037570051X</a>
评论 #34810092 未加载
neilv超过 2 年前
How do you get all the Clippies to cooperate towards a collective goal?<p>Asking for a friend.
评论 #34812886 未加载
A4ET8a8uTh0超过 2 年前
It made me smile.<p>I recently thought about why certain options are restricted from GPT ( predictions come to mind ) and it slowly became apparent that with enough information you could predict not a specific individual making a specific move, but likely specific event happening.<p>Would you like to proceed?
jvanderbot超过 2 年前
A less dystopian version can be had in the Bobiverse series. Love those books!
评论 #34823148 未加载