TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

GPT-4 “discovered” the same sorting algorithm as AlphaDev by removing “mov S P”

190 点作者 machdiamonds将近 2 年前

18 条评论

danpalmer将近 2 年前
The paper shows several distinct improvements to a sorting algorithm, and presents evidence that the process is generally applicable. This tweet points GPT-4 at 20 instructions and asks if any can be removed, and it finds one optimisation.<p>That&#x27;s good to see from GPT-4, but the comparison seems disingenuous, and I&#x27;d expect more from someone in academia.
评论 #36255553 未加载
评论 #36249329 未加载
评论 #36251236 未加载
评论 #36256908 未加载
评论 #36250215 未加载
评论 #36272183 未加载
评论 #36256642 未加载
cafaxo将近 2 年前
GPT-4&#x27;s explanation of its optimization does not make sense to me. It writes &quot;Instead of moving it to P, we can directly use S in the following comparisons, saving one instruction.&quot; but then proceeds to use P as if that mov had happened.<p>AlphaDev&#x27;s optimization relies on the fact that B and C are already in the correct order. This precondition is missing from the prompt given to GPT-4. It seems that GPT-4 is hallucinating something that only resembles the correct optimization at first glance.
评论 #36251199 未加载
评论 #36250911 未加载
评论 #36251241 未加载
pfedak将近 2 年前
I posted a reply, but I think ultimately this is just a coincidence. There&#x27;s a naïve reason that &quot;mov S P&quot; looks redundant (&quot;just use S instead of P later&quot;), but in typical GPT fashion, this is specious, and can&#x27;t actually be done. It&#x27;s essentially trying to swap two variables without using a temporary. If x86 had a conditional swap instruction, it could, but it doesn&#x27;t, and just doing &quot;cmp,cmov,cmov&quot; can&#x27;t handle it.<p>Another giveaway is that removing that line in the real optimization changes the output of the provided snippet if C &lt; B. It feels like a hard sell to say GPT picked this line for that subtle reason based on information not provided, but explained it with something only correct at surface level.
评论 #36252720 未加载
beisner将近 2 年前
Is it possible that GPT-4 already had this new optimization somewhere in its training set? The optimizing patch DeepMind published on has been floating around for several months at this point…<p>Edit: I found the merges for sorting (Jan 2022) [1] and hashing (Jan 2023) [2]. Both of these are very plausibly in the training set for GPT-4, which was frozen sometime in March 2023.<p>[1] <a href="https:&#x2F;&#x2F;reviews.llvm.org&#x2F;D118029" rel="nofollow">https:&#x2F;&#x2F;reviews.llvm.org&#x2F;D118029</a><p>[2] <a href="https:&#x2F;&#x2F;github.com&#x2F;abseil&#x2F;abseil-cpp&#x2F;commit&#x2F;74eee2aff683cc7dcd2dbaa69b2c654596d8024e">https:&#x2F;&#x2F;github.com&#x2F;abseil&#x2F;abseil-cpp&#x2F;commit&#x2F;74eee2aff683cc7d...</a>
评论 #36250536 未加载
评论 #36249248 未加载
评论 #36250549 未加载
awegio将近 2 年前
DeepMind&#x27;s blog post on AlphaDev says:<p>&gt; AlphaDev uncovered faster algorithms by starting from scratch rather than refining existing algorithms<p>Finding that specific optimization, especially when given the comments, seems almost trivial by comparison.<p>Edit: I tried to understand the optimization in question. This is not the full sort3 algorithm, but only under the assumption that B &lt; C. In that case the GPT-4 answer is actually wrong because it wasn&#x27;t given that assumption.
imranq将近 2 年前
This is an impressive demonstration of GPT-4&#x27;s logic and reasoning capabilities, but the DM result uses a real-world reward signal (e.g. the list is sorted correctly) to validate its results whereas we can never be sure that GPT-4 outputs are hallucinated or not. Also this experiment would never be done if not for the DM paper in the first place.
评论 #36249714 未加载
cypherpunks01将近 2 年前
Wow no reinforcement learning needed! Amazing to think, all they would&#x27;ve had to do instead is spend a decade or two building towards a 1 trillion parameter transformer model and spend 100m or two training it. Then fine-tune the model using.. y&#x27;know nevermind : )
bradley13将近 2 年前
Trying to attribute intelligent intent where there is none. ChatGPT just puts tokens in order, following amalgamated examples it has seen.<p>When it screws up, we call it a hallucination. With careful prompting, you can get it to screw up in a way that works.<p>Color me unimpressed.
评论 #36256491 未加载
评论 #36257231 未加载
agluszak将近 2 年前
Can you set ChatGPT&#x27;s temperature by simply asking it to &quot;use temperature 0.0&quot;? Doesn&#x27;t sound legit
评论 #36257193 未加载
yieldcrv将近 2 年前
I&#x27;ve gotten extremely odd algorithms and solutions that saved me lots of resources.<p>Lots of bitshifting and unsigned ints and approximations these days. Stuff I would have never thought of, and then I can talk it through how something else only I thought of would be even <i>more</i> applicable, and it refines it for that too! Great pair programmer!<p>This level of competence will never show up in a live no-resource interview or any (watched) time trial I do.
评论 #36254566 未加载
bitshiftfaced将近 2 年前
Since they brute forced all solution programs for this case anyway, Deepmind already admitted that <i>they</i> didn&#x27;t even need to use AlphaDev in this particular case. GPT-4 being able to work it out is great, but it says nothing about the value of what the AlphaZero&#x2F;Dev algorithm and the kinds of interesting problems you can solve with it.
summerlight将近 2 年前
Isn&#x27;t it something like some interviewer who already knows the answer asks some interviewee a very specific question designed with a certain level of guidance in mind? It&#x27;s still impressive for LLM, but it&#x27;s not even apple-to-orange comparison.
yb303将近 2 年前
It didn&#x27;t discover anything. It only minces words it found online up to 2021.
评论 #36255321 未加载
IshKebab将近 2 年前
Very impressive, though you are using a lot of prior knowledge to guide it towards the solution you already know. The RL version presumably wasn&#x27;t given the hint that it could remove an instruction.<p>Still very impressive, but I&#x27;d like to see it work on some other assembly for which the &quot;answer&quot; isn&#x27;t already known.
world2vec将近 2 年前
The green GPT icon in the chat would indicate it&#x27;s GPT-3.5 right? GPT-4 is purple, at least for me.
评论 #36251310 未加载
globular-toast将近 2 年前
Isn&#x27;t it cheating a bit to prompt for removing a line? Not all algorithms can be optimised by removing a single step.
dang将近 2 年前
Comments moved to <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=36228125" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=36228125</a>.
评论 #36252564 未加载
评论 #36252360 未加载
BulgarianIdiot将近 2 年前
I don&#x27;t like the implications of the quoted &quot;discovered&quot;.
评论 #36249377 未加载