TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Accidentally quadratic: When Python is faster than C++

218 点作者 mehrdadn大约 4 年前

14 条评论

mehrdadn大约 4 年前
If you&#x27;re wondering whether this is a theoretical or practical problem: I actually observed some of this effect in practice first, and only after thinking about it for a while did the larger issue (and the complexity implications) dawn on me.<p>I had something like a set&lt;string&gt; or a set&lt;set&lt;string&gt;&gt; (or map... I can&#x27;t remember which) somewhere in my program a few years ago, and I was trying to improve the program&#x27;s performance. I tried breaking into it several times and found it quite perplexing that the bottleneck appeared to be the set&lt;&gt; container. I mean, I get that cache locality and all has an effect, but it seemed to be having a little too much of an effect. Why did I find this odd? Because other languages (like C#) have tree-based sets and maps too, but I&#x27;d never felt they were quite as slow as I was seeing them in C++. So I felt something weird must be going on.<p>I tried to step through the program for a while and observe what&#x27;s going on, and at some point, I realized (perhaps I hit the same breakpoint twice? I don&#x27;t recall) that these functions were being called more often than I intuitively thought would be necessary. Which was obvious in hindsight, as less() needs to be called <i>multiple times on the same objects</i> (4 times at level 2). Now, this hadn&#x27;t resulted in <i>quadratic</i> behavior, but that was only because my data structures weren&#x27;t arbitrarily deep—the slowdown was nevertheless a significant constant factor at the core of my program, only linear because the data structure&#x27;s depth was bounded.<p>So once I had realized the implications of this, including that constant-factor differences can actually turn into polynomial ones, I eventually decided to post an article about it on arXiv... hence this article. Aside from the complexity issue illustrated in the article, one of my (unexpected) higher-level takeaways was that <i>you can&#x27;t just take any utility function in a program and blindly put it into a library</i>: even if it&#x27;s correct, you probably have hidden assumptions about its use cases that won&#x27;t hold true in general. You really need to think about how it could be used in ways you didn&#x27;t initially expect, and this may well require a different kind of code review with an entirely different mindset than before. It really drove home the point for me that writing a library is quite a bit different (and in some ways more difficult) than writing an application.<p>It&#x27;s possible there is more theory lying underneath this that I haven&#x27;t touched on—it would be interesting if someone can take this and find more interesting consequences of it. For example, maybe when analyzing algorithms we need to consider something akin to a &quot;recursive time complexity&quot;, too? Is there another useful notion of complexity that we can generalize here?<p>Anyway, hope people enjoy reading it and take something useful away from it. :)
评论 #26342505 未加载
评论 #26340897 未加载
评论 #26340926 未加载
a-dub大约 4 年前
cpython is faster than c++ under the following circumstances:<p>1) when you&#x27;re using native libraries via python that were written by better c&#x2F;c++ programmers than you are and you&#x27;re spending most of your time within them<p>2) when you&#x27;re using native libraries in python that are better implementations than the c&#x2F;c++ libraries you&#x27;re comparing against<p>3) when you don&#x27;t know the libraries you&#x27;re using in c&#x2F;c++ (what they&#x27;re talking about here)<p>...otherwise, if you&#x27;re just using doing basic control flow optimizing compiler c&#x2F;c++ will almost always be faster. (unless you&#x27;re using numba or pypy or something).<p>point stands about the constants though. yes, asymptotic analysis will tell you about how an algorithm&#x27;s performance scales, but if you&#x27;re just looking for raw performance, especially if you have &quot;little data&quot;, you really have to measure as implementation details of the software and hardware start to have a greater effect. (often times the growth of execution time isn&#x27;t even smooth with the growth of n for small n)
评论 #26343009 未加载
评论 #26341430 未加载
评论 #26340631 未加载
IgorPartola大约 4 年前
I’ve been thinking about this lately. Can you actually be faster than C? Like, in the sense that you can transpile any bit of Python or Lisp or Haskell or Rust or JS into C but the opposite isn’t necessarily true because not all those language support all the features exposed in C (such as goto, no bounds checking, pointer arithmetic, etc.), any algorithm for say parsing JSON can be expressed equally as efficiently in C, while a clever and hacky C-specific algorithm cannot necessarily be expressed in those higher level languages.<p>In other words, is “faster than C” even a good metric if all it means that “if you implement something inefficiently in C it will be faster than if it is implemented efficiently in not-C”?
评论 #26339022 未加载
评论 #26338899 未加载
评论 #26339069 未加载
评论 #26338944 未加载
评论 #26339035 未加载
评论 #26338850 未加载
评论 #26338858 未加载
评论 #26338862 未加载
评论 #26343494 未加载
评论 #26342452 未加载
评论 #26338765 未加载
评论 #26342075 未加载
评论 #26339693 未加载
评论 #26344071 未加载
评论 #26340290 未加载
评论 #26345451 未加载
评论 #26338860 未加载
评论 #26340525 未加载
评论 #26342538 未加载
评论 #26338788 未加载
评论 #26340530 未加载
评论 #26338845 未加载
评论 #26338808 未加载
nneonneo大约 4 年前
Hang on, this isn&#x27;t accidentally quadratic, this is <i>accidentally exponential</i> in the worst case. If you set the tree branching factor to one, then the running time of C++‘s std::less becomes 2^h, but there’s only h nodes.<p>Demo: <a href="https:&#x2F;&#x2F;ideone.com&#x2F;CBIEAE" rel="nofollow">https:&#x2F;&#x2F;ideone.com&#x2F;CBIEAE</a><p>This creates ~60 objects, but takes something like 2^30 operations to resolve (it times out on this online runner, and takes around 5s on my laptop with -O3).<p>That&#x27;s <i>much</i> worse than claimed in this paper! An accidentally-exponential algorithm is the kind of thing that makes DoS attacks trivial...
评论 #26347836 未加载
leecarraher大约 4 年前
This is a pretty clickbaity title in my opinion. Bubble sort in lower level language X is going to be slower than quick sort in high level language Y. And bubblesort in high level language is going to be faster than merge sort for small data sets on low level language X. If you aren&#x27;t comparing the same underlying routine, or data application, I don&#x27;t think any conclusion should be made. Comparisons between languages is exactly why asymptomatic analysis was devised. Extract the process from the low level and hardware characteristics, and get the overall complexity growth. But this doesn&#x27;t work the other way around. You can&#x27;t compare different routines in different languages and expect big oh to be comparable.
评论 #26343901 未加载
frankus大约 4 年前
Here’s an article from 2001 discussing a very similar issue: <a href="https:&#x2F;&#x2F;www.joelonsoftware.com&#x2F;2001&#x2F;12&#x2F;11&#x2F;back-to-basics&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.joelonsoftware.com&#x2F;2001&#x2F;12&#x2F;11&#x2F;back-to-basics&#x2F;</a>
derriz大约 4 年前
Am I missing something? In the paper the definition of cmp3 on page 2 seems to have a bug - as defined, wouldn&#x27;t cmp3([1,2,3], [1]) return 0?<p><pre><code> # Uses 3-way cmp() for primitives def cmp3(a, b): if not isinstance(a, list): global c; c += 1 return cmp(a, b) for x, y in zip(a, b): r = cmp3(x, y) if r != 0: return r return 0 </code></pre> This isn&#x27;t generally the expected behaviour for comparing lists, surely?
评论 #26340109 未加载
评论 #26340117 未加载
froh大约 4 年前
If I read the paper correctly, then it compares three-way comparison with two two-way comparisons, for a recursively defined (tree) data type.<p>The paper points out that the convenience of just defining &quot;less than&quot;, and heaving &quot;equals&quot; derived from that can be costly. Specifically, for the recursively defined data structure (tree), a three-way comparison which is derived canonically from the two way compare <i>seems</i> to entail not a linear but an quadratic number of comparisons.<p>What I don&#x27;t understand is what is happening in &#x27;lt2&#x27;.<p>this is what I&#x27;d expect for __lt__<p><pre><code> lt(a, b) also known as (a __lt__ b) is returning True, iff a &lt; b, elementwise for lists False otherwise for same length lists. </code></pre> I also do understand cmp2.<p><pre><code> (a __eq__ b) iff not (a __lt__ b) and not (b __lt__ a) </code></pre> so looking at<p><pre><code> cmp(a, b) = lt(a, b) - lt(b, a) </code></pre> I get<p><pre><code> a &lt; b: 1 - 0 ==&gt; 1 b &lt; a: 0 - 1 ==&gt; -1 a == b: 0 - 0 ==&gt; 0 </code></pre> which makes sense.<p>Now two questions arise with respect to the presented hypothesis and the paper:<p>1. why does the paper call lt2 twice, recursively?<p>2. why does the paper compare the performance of their lt2 and lt3 instead of the performance of cmp2 and cmp3?<p>I intuit, when taking the double recursion out of lt2, which imnsho is erroneous, and when comparing cmp2 and cmp3, we&#x27;ll see a performance penalty of a factor 2, between cmp2 and cmp3, and identical run times for lt2 and lt3, as it should be.<p>What am I missing?<p>edit: updated for clarity
评论 #26343054 未加载
评论 #26348293 未加载
brundolf大约 4 年前
Reminds me of the sscanf thing that popped up a few days ago (in fact I assumed this was about that at first): <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=26302744" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=26302744</a><p>I wonder (genuinely asking, not being snarky) what it is about C&#x2F;C++ that seems to make these issues more common? It&#x27;s also possible my perception of &quot;more common&quot; has just been inflated by seeing multiple examples in a single week
评论 #26344903 未加载
brobdingnagians大约 4 年前
Very interesting. I think it makes C++ come across looking quite good. The committee has considered a case where C++ comparisons are lacking, and C++20 rectifies the situation by having more expressive comparisons with partial and total orderings. C++ may be slower than other languages in doing innovations, but they move relentlessly forward with trying to adopt a good way of accomplishing new features &amp; rectifying mistakes from the past.
评论 #26348664 未加载
ac130kz大约 4 年前
The code doesn&#x27;t compile, i.e. is in the broken state. The author is missing the comparison operators.<p><a href="https:&#x2F;&#x2F;godbolt.org&#x2F;z&#x2F;5EWhec" rel="nofollow">https:&#x2F;&#x2F;godbolt.org&#x2F;z&#x2F;5EWhec</a>
评论 #26344971 未加载
halayli大约 4 年前
How can this paper be taken seriously when the paper doesn&#x27;t even show the compilation flags?
评论 #26339757 未加载
评论 #26339692 未加载
david2ndaccount大约 4 年前
Have academic CS articles always had click-bait titles?
评论 #26338976 未加载
评论 #26339356 未加载
globular-toast大约 4 年前
Why are we back to learning basic computer science? This isn&#x27;t news to anyone here is it?
评论 #26341542 未加载
评论 #26340914 未加载
评论 #26340802 未加载