TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Book Review: Reframing Superintelligence

52 点作者 nikbackm超过 5 年前

9 条评论

glenstein超过 5 年前
The claim is apparently that superhuman, savant-level intelligence could be confined to a domain of knowledge and not risk becoming generalized intelligence. I&#x27;m skeptical.<p>If you&#x27;re &quot;only&quot; superintelligent at language translation, or writing movies, or chess, I suspect as we ascend the tiers of increasingly super superingelligences, there&#x27;s a depth of informational, structural understanding that avails itself of abstract meta principles, and meta-meta principles, and meta-meta-meta principles, and on to infinity. And that at a sufficiently high level of abstraction, something about be brilliant at translating the subtle irony of a Shakespearian sonnet into a dead tribal language is also at play in weighing strategic options in an incredibly complicated game of chess, is also at play in reading culture and finding out what kind of movie will be most successful at the box-office.<p>I think any domain-specific intelligence, as it approximates &quot;perfect&quot;, would independently discover and solve similar high level questions and be transferable to other domains, the way there are general principles of manufacturing that apply to multiple products. And from a sufficiently advanced perspective, &quot;solving&quot; chess and &quot;solving&quot; Shakespearan sonnet translation would look as similar to each other and panting a car red vs painting a car blue.
评论 #20824461 未加载
评论 #20825143 未加载
评论 #20824993 未加载
评论 #20826940 未加载
评论 #20824799 未加载
andreyk超过 5 年前
See also this summary of the report: <a href="https:&#x2F;&#x2F;singularityhub.com&#x2F;2019&#x2F;06&#x2F;02&#x2F;less-like-us-an-alternate-theory-of-artificial-general-intelligence&#x2F;" rel="nofollow">https:&#x2F;&#x2F;singularityhub.com&#x2F;2019&#x2F;06&#x2F;02&#x2F;less-like-us-an-altern...</a><p>The argument makes a lot of sense to me as an AI researcher; the idea that we will somehow get to self improving agents that can do any and all things they want to maximize the output of paper clips (as in the famous Bostrom thought experiment) is miles away from how AI is done today in practice AND how software functions. I suspect most software engineers, especially with knowledge of AI, would find the &#x27;AI service&#x27; idea which actually reflects how AI is done today much more plausible and worth worrying about than the Bostom science fiction-y fears of AGI...
评论 #20824842 未加载
mnemonicsloth超过 5 年前
I don&#x27;t understand why people are so worried about superintelligence. Moore&#x27;s law is dying. Computers are not going to get much faster, or much cheaper. Parallelization is going to be the only way to get more powerful computers and there are real, possibly insurmountable, limits on how well you can do that. The computing sector is going to look like aerospace: the Boeing 737 of today is better in some ways, but it still looks almost exactly like the Boeing 737 of 50 years ago
评论 #20826102 未加载
评论 #20824601 未加载
评论 #20825735 未加载
codeisawesome超过 5 年前
&gt; But in the end, it would just be a translation app. It wouldn’t want to take over the world. It wouldn’t even “want” to become better at translating than it was already. It would just translate stuff really well.<p>&gt; It could have media services that can write books or generate movies to fit your personal tastes.<p>I&#x27;m not an expert in this topic, but wouldn&#x27;t you say that the ability to create compelling narratives (even more so than &#x27;mechanical&#x27; translation between languages), pretty much <i>relies</i> on your ability to empathise at some level with people who want to take over the world or the people who want to stop them?<p>How would AI come up with the plot for one of the most profitable franchises in recent history - &quot;Avengers: Infinity War&quot; movies? It would have to be programmed with an understanding for Thanos&#x27; perspective, and the fundamental will of most everyone around him to be terrified of that and disagree - and even then it&#x27;s not a good movie franchise without the romance and familial dynamics between so many characters.<p>If an AI can already understand all that (and <i>know</i> that it has to understand all that) - well you&#x27;ve created a pretty smart human already - and the OP&#x27;s argument about differentiating these powers doesn&#x27;t seem to hold...
Upvoter33超过 5 年前
I&#x27;ve never been very interested in the Bostrom view of AI, as the viewpoint there seems to come from someone fairly removed from how computer programming works. It&#x27;s always felt more like an intellectual exercise rather than very grounded in practicality (but that&#x27;s just my two cents).<p>Where I got more interested in Bostrom was some notions I heard from him (I think) on the general nature of science. I&#x27;d always assumed that learning more about how nature actually works was a strict positive, but now am convinced (more than ever) that we are just in a race to discover a technology that will harm us all (we&#x27;ve found some already, but who knows what worse discoveries are out there?)
etxm超过 5 年前
Bostrom also wrote a paper on The Simulation Argument that was pretty awesome[1]. It was referenced recently in The End of the World podcast. [2]<p>1- <a href="https:&#x2F;&#x2F;www.simulation-argument.com&#x2F;simulation.pdf" rel="nofollow">https:&#x2F;&#x2F;www.simulation-argument.com&#x2F;simulation.pdf</a> 2- <a href="https:&#x2F;&#x2F;www.theendwithjosh.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.theendwithjosh.com&#x2F;</a>
prvc超过 5 年前
I see no a priori reason why &quot;superintelligent services&quot; as defined in the review would necessarily be inherently safer, due to the complexity of the systems they affect, and the possibility of emergent effects that might be in some sense equivalent to the hypothetical effects of an &quot;agent type&quot; AI. I also think that the concept needs more elaboration for that very reason. Furthermore, it could be that there is a &quot;recipe&quot; for forming general AI by integrating a small number of &quot;service type&quot; AIs, assuming that term refers to a real concept in the first place.
scottlocklin超过 5 年前
A review of a book by a serial fabulist (Drexler) compared to that of a bozo moonlighting as a science fiction writer (Bostrom) done by a psychologist on a subject none of them have the slightest whit of a clue about.<p>&gt; All of this seems kind of common sense to me now. This is worrying, because I didn’t think of any of it when I read Superintelligence in 2014<p>Dunning Kruger is something that should come to mind here, doctor. People who know a decision tree from an echo state network kind of saw that as being incredibly dumb when it came out.<p>What has happened in the last 5 years isn&#x27;t that the field has matured; it&#x27;s as gaseous and filled with prevaricating marketers, science fiction hawking twits and overt mountebanks as ever. The difference is, 5 years later, rather than the swingularity-like super explosion of exponential increase in human knowledge, we&#x27;re actually just as dumb as we were 5 years ago when we figured out how to classify German traffic signs, and we have slightly better libraries than we used to. No great benefit to the human race has come of &quot;AI&quot; -and nothing resembling &quot;AI&quot; or any kind of &quot;I&quot; has even hinted of its existence. In another 5 years I&#x27;d venture a guess machine learning will remain about as useful as it is now, which is to say, with no profitable companies based on &quot;AI,&quot; let alone replacing human intelligences anywhere. And we&#x27;ll sadly probably still have yoyos like Hanson, Drexler and Yudkowsky lecturing us on how to deal with this nonexistent threat.<p>Meanwhile, the actual danger to our society is surveillance capitalism and government agencies using dumb ass analytics related to singular value decomposition. Nobody wants to talk about this, presumably because it&#x27;s real and we&#x27;d have to make difficult choices as a society to deal with it. Easier and more profitable to wank about Asimovian positronic brain science fiction.
评论 #20824668 未加载
falcor84超过 5 年前
I was really surprised that there was no mention of APIs there. Obviously as &quot;services&quot; many (most?) of these AI services would be available via APIs. There are already machine-readable directories of APIs and AI services relying on external APIs, so we can extrapolate that we&#x27;ll see more and more AI&#x2F;ML systems experimenting with various external APIs as part of their learning.<p>From this perspective, it&#x27;s very clear to me that there&#x27;s a big difference between a translation service and a service that would &quot;steer Fortune 500 companies&quot;. The latter will be much more open-ended and most likely dynamically rely on many other services. Indeed, I would also expect it to rely on artificial-artificial-intelligence services such as Mechanical Turk and Upwork, giving it a lot more flexibility.<p>What is to prevent that complex service from evolving beyond its creator&#x27;s expectations?