TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

{n} times faster than C

447 pointsby 414owenalmost 2 years ago

31 comments

torstenvlalmost 2 years ago
I&#x27;m not so sure that the right take-away is &quot;hand-written assembler is 6x faster than C.&quot; It&#x27;s more like &quot;jumps are a lot slower than conditional arithmetic.&quot; And that can [edit:often] be achieved easily in C by simply not using switch statements when an if statement or two will work fine.<p>Rewriting the C function as follows got a 5.5x speedup:<p><pre><code> int run_switches(char *input) { int r = 0; char c; while (1) { c = *input++; if (c == &#x27;s&#x27;) r++; if (c == &#x27;p&#x27;) r--; if (c == &#x27;\0&#x27;) break; } return r; } </code></pre> Results:<p><pre><code> [16:50:14 user@boxer ~&#x2F;looptest] $ gcc -O3 bench.c loop1.c -o lone [16:50:37 user@boxer ~&#x2F;looptest] $ gcc -O3 bench.c loop2.c -o ltwo [16:50:47 user@boxer ~&#x2F;looptest] $ time .&#x2F;lone 1000 1 449000 .&#x2F;lone 1000 1 3.58s user 0.00s system 99% cpu 3.589 total [16:50:57 user@boxer ~&#x2F;looptest] $ time .&#x2F;ltwo 1000 1 449000 .&#x2F;ltwo 1000 1 0.65s user 0.00s system 99% cpu 0.658 total</code></pre>
评论 #36623958 未加载
评论 #36623708 未加载
评论 #36623180 未加载
评论 #36624329 未加载
评论 #36623284 未加载
评论 #36628710 未加载
评论 #36625110 未加载
评论 #36631636 未加载
评论 #36627638 未加载
nwallinalmost 2 years ago
IMHO the original code wasn&#x27;t written in a way that&#x27;s particularly friendly to compilers. If you write it like this:<p><pre><code> int run_switches_branchless(const char* s) { int result = 0; for (; *s; ++s) { result += *s == &#x27;s&#x27;; result -= *s == &#x27;p&#x27;; } return result; } </code></pre> ...the compiler will do all the branchless sete&#x2F;cmov stuff as it sees fit. It will be the same speed as the optimized assembly in the post, +&#x2F;- something insignificant. However it won&#x27;t unroll and vectorize the loop. If you write it like this:<p><pre><code> int run_switches_vectorized(const char* s, size_t size) { int result = 0; for (; size--; ++s) { result += *s == &#x27;s&#x27;; result -= *s == &#x27;p&#x27;; } return result; } </code></pre> It will know the size of the loop, and will unroll it and use AVX-512 instructions if they&#x27;re available. This will be substantially faster than the first loop for large inputs, although I&#x27;m too lazy to benchmark just how much faster it is.<p>Now, this requires knowing the size of your string in advance, and maybe you&#x27;re the sort of C programmer who doesn&#x27;t keep track of how big your strings are. I&#x27;m not your coworker, I don&#x27;t review your code. Do what you want. But you really <i>really</i> probably shouldn&#x27;t.<p><a href="https:&#x2F;&#x2F;godbolt.org&#x2F;z&#x2F;rde51zMd8" rel="nofollow noreferrer">https:&#x2F;&#x2F;godbolt.org&#x2F;z&#x2F;rde51zMd8</a>
评论 #36623823 未加载
评论 #36627023 未加载
评论 #36625584 未加载
评论 #36623408 未加载
Const-mealmost 2 years ago
I’m probably an optimization expert, and I would solve that problem completely differently.<p>On my computer, the initial C version runs at 389 MB &#x2F; second. I haven’t tested the assembly versions, but if they deliver the same 6.2x speedup, would result in 2.4 GB&#x2F;second here.<p>Here’s C++ version which for long buffers exceeds 24 GB&#x2F;second on my computer: <a href="https:&#x2F;&#x2F;gist.github.com&#x2F;Const-me&#x2F;3ade77faad47f0fbb0538965ae7f8e04" rel="nofollow noreferrer">https:&#x2F;&#x2F;gist.github.com&#x2F;Const-me&#x2F;3ade77faad47f0fbb0538965ae7...</a> That’s 61x speedup compared to the original version, without any assembly, based on AVX2 intrinsics.
评论 #36624746 未加载
评论 #36624301 未加载
评论 #36626491 未加载
评论 #36626232 未加载
Sesse__almost 2 years ago
This code screams for SIMD! If you can change the prototype to take an explicit length, you could easily read and process 16 bytes at a time (the compares will give you values you can just add and subtract directly). Heck, even calling strlen() at the function&#x27;s start to get the explicit length would probably be worth it.
camel-cdralmost 2 years ago
I threw together a quick risc-v vectorized implementation:<p><pre><code> size_t run(char *str) { uint8_t *p = (uint8_t*)str; long end = 0; size_t res = 0, vl; while (1) { vl = __riscv_vsetvlmax_e8m8(); vuint8m8_t v = __riscv_vle8ff_v_u8m8(p, &amp;vl, vl); end = __riscv_vfirst_m_b1(__riscv_vmseq_vx_u8m8_b1(v, &#x27;\0&#x27;, vl), vl); if (end &gt;= 0) break; res += __riscv_vcpop_m_b1(__riscv_vmseq_vx_u8m8_b1(v, &#x27;s&#x27;, vl), vl); res -= __riscv_vcpop_m_b1(__riscv_vmseq_vx_u8m8_b1(v, &#x27;p&#x27;, vl), vl); p += vl; } vl = __riscv_vsetvl_e8m8(end); vuint8m8_t v = __riscv_vle8_v_u8m8(p, vl); res += __riscv_vcpop_m_b1(__riscv_vmseq_vx_u8m8_b1(v, &#x27;s&#x27;, vl), vl); res -= __riscv_vcpop_m_b1(__riscv_vmseq_vx_u8m8_b1(v, &#x27;p&#x27;, vl), vl); return res; } </code></pre> Here are the results from the above, the switch and the table c implementation, ran on my mangopi mq pro (C906, in order rv64gc with rvv 0.7.1, and a 128 bit vector length):<p><pre><code> switch: 0.19 Bytes&#x2F;Cycle tbl: 0.17 Bytes&#x2F;Cycle rvv: 1.57 Bytes&#x2F;Cycle (dips down to 1.35 after ~30 KiB) </code></pre> Edit: you can go up to 2&#x2F;1.7 Bytes&#x2F;Cycle, if you make sure the pointer is page aligned (and vl isn&#x27;t larger than the page size), see comments
评论 #36623156 未加载
okaleniukalmost 2 years ago
I think, it&#x27;s a particular quirk of x86 architecture. Branching is expensive in comparison because not doing branching is super cheap. <a href="https:&#x2F;&#x2F;wordsandbuttons.online&#x2F;challenge_your_performance_intuition_with_cpp_operators.html" rel="nofollow noreferrer">https:&#x2F;&#x2F;wordsandbuttons.online&#x2F;challenge_your_performance_in...</a><p>However, on other processors, this might not be the case. <a href="https:&#x2F;&#x2F;wordsandbuttons.online&#x2F;using_logical_operators_for_logical_operations_is_good.html" rel="nofollow noreferrer">https:&#x2F;&#x2F;wordsandbuttons.online&#x2F;using_logical_operators_for_l...</a><p>The good question is what do we need C for in general? Of course, we can hand-tailor our code to run best on one particular piece of hardware. And we don&#x27;t need C for that, it would be the wrong tool. We need assembly (and a decent macro system for some sugar)<p>But the original goal of C was to make translating system-level code from one platform to another easier. And we&#x27;re expected to lose efficiency on this operation. It&#x27;s like instead of writing a poem in Hindi and translating it in Urdu, we write one in Esperanto and then translate to whatever language we want automatically. You don&#x27;t get two brilliant poems, you only get two poor translations, but you get them fast. That&#x27;s what C is for.
eklitzkealmost 2 years ago
Rearranging branches (and perhaps blocks too?) will definitely be done if you are building using FDO, because without FDO (or PGO) the compiler has no idea how likely each branch is to be taken. Cmov can also be enabled by FDO in some cases.<p>However, whether or not using cmov is effective compared to a regular test&#x2F;jump is highly dependent on how predictable the branch is, with cmov typically performing better when the branch is very unpredictable. Since they got a 6x speedup with cmov, I assume that their test input (which isn&#x27;t described in the post, and is also not in their GitHub repo) consists of random strings consisting almost entirely of s and p characters. There&#x27;s nothing wrong with this, but it does make the post seem a little misleading to me, as their clever speedup is mostly about exploiting an unmentioned property of the data that is highly specific to their benchmark.
评论 #36624125 未加载
评论 #36623739 未加载
评论 #36624658 未加载
xoranthalmost 2 years ago
I think I managed to improve on both this post, and its sequel, at the cost of specializing the function for the case of a string made only of &#x27;s&#x27; and &#x27;p&#x27;.<p>The benchmark only tests strings made of &#x27;s&#x27; and &#x27;p&#x27;, so I think it is fair.<p>The idea is as follow. We want to increase `res` by one when the next character is `s`. Naively, we might try something like this:<p><pre><code> res += (c - &#x27;r&#x27;); &#x2F;&#x2F; is `res += 1` when c == &#x27;s&#x27; </code></pre> This doesn&#x27;t work, as `&#x27;p&#x27; - &#x27;r&#x27; == -2`, and we&#x27;d need it to be -1.<p>But `&#x27;p&#x27; - &#x27;r&#x27;`, when viewer as an unsigned integer, underflows, setting the carry flag. Turns out x64 has an instruction (adc) that adds two registers _plus_ the carry flag.<p>Therefore we can replace two `cmp, cmov` with one `sub, adc`:<p><pre><code> run_switches: xor eax, eax # res = 0 loop: movsx ecx, byte ptr [rdi] test ecx, ecx je ret inc rdi sub ecx, &#x27;r&#x27; adc eax, ecx # Magic happens here jmp loop ret: ret </code></pre> Benchmarks are as follows (`bench-x64-8` is the asm above):<p><pre><code> Summary &#x27;01-six-times-faster-than-c&#x2F;bench-x64-8 1000 1&#x27; ran 1.08 ± 0.00 times faster than &#x27;02-the-same-speed-as-c&#x2F;bench-c-4-clang 1000 1&#x27; 1.66 ± 0.00 times faster than &#x27;01-six-times-faster-than-c&#x2F;bench-x64-7 1000 1&#x27; </code></pre> Of course, one could improve things further using SWAR&#x2F;SIMD...
评论 #36624157 未加载
评论 #36628447 未加载
sltkralmost 2 years ago
How much faster is this:<p><pre><code> int run_switches(const char *buf) { size_t len = strlen(buf); int res = 0; for (size_t i = 0; i &lt; len; ++i) { res += (buf[i] == &#x27;s&#x27;) - (buf[i] == &#x27;p&#x27;); } return res; } </code></pre> strlen() should be implemented in a pretty fast way, and after the buffer size is known, the compiler can autovectorize the inner loop, which does happen in practice: <a href="https:&#x2F;&#x2F;gcc.godbolt.org&#x2F;z&#x2F;qYfadPYoq" rel="nofollow noreferrer">https:&#x2F;&#x2F;gcc.godbolt.org&#x2F;z&#x2F;qYfadPYoq</a>
aidenn0almost 2 years ago
A while back, I wrote a UTF-8 decoder in Common Lisp, targeting SBCL (it already has one built in, this was an exercise). Pretty much all of the optimization win (after the obvious low-hanging fruit) was structuring the code so that the compiler would generate cmov* instructions rather than branches.
评论 #36622681 未加载
评论 #36623188 未加载
fefe23almost 2 years ago
First, before optimizing you should consider correctness and security. input should be const and the return value should be ssize_t (so you don&#x27;t have numeric overflow on 64-bit).<p>Second, consider this replacement function:<p><pre><code> ssize_t test(const char \*input) { ssize_t res = 0; size_t l = strlen(input); size_t i; for (i=0; i&lt;l; ++i) { res += (input[i] == &#x27;s&#x27;) - (input[i] == &#x27;p&#x27;); } return res; } </code></pre> The timings are (using gcc -O3 -march=native): your function 640 cycles, mine 128 cycles. How can that be? I&#x27;m reading the memory twice! I have one call to strlen in there, and memory is slow. Shouldn&#x27;t this be much slower?<p>No. strlen is a hack that uses vector instructions even though it may technically read beyond the string length. It makes sure not to cross page boundaries so it will not cause adverse reactions, but valgrind needs a suppression exception to not complain about it.<p>If you know the length beforehand, the compiler can vectorize and unroll the loop, which it happens to do here. To great effect, if I may say so.<p>The art of writing fast code is usually to get out of the way of the compiler, which will do a perfectly fine job if you let it.<p>If you really wanted to, you could get rid of the strlen by hacking your logic into what strlen does. That would make the C code much less readable and not actually help that much. My test string is &quot;abcdefghijklmnopqrstuvxyz&quot;, so it&#x27;s all in the l1 cache.
torstenvlalmost 2 years ago
There&#x27;s an error in the pseudocode.<p><pre><code> cmp ecx, &#x27;s&#x27; # if (c == &#x27;s&#x27;) jne loop # continue add eax, 1 # res++ jmp loop # continue </code></pre> should be<p><pre><code> cmp ecx, &#x27;s&#x27; # if (c != &#x27;s&#x27;) jne loop # continue add eax, 1 # res++ jmp loop # continue</code></pre>
评论 #36622987 未加载
评论 #36623288 未加载
414owenalmost 2 years ago
A clickbait title for an in-depth look at hand-optimizing a very simple loop.
评论 #36623217 未加载
评论 #36634125 未加载
jtrianglealmost 2 years ago
It&#x27;s a cardinal rule that any time someone utters &quot;XYZ is n faster than C&quot; someone comes along and shows C is actually 2x faster than XYZ.
评论 #36625176 未加载
BoppreHalmost 2 years ago
You can also use math to avoid most of the jumps:<p><pre><code> int run_switches(char *input) { int res = 0; while (true) { char c = *input++; if (c == &#x27;\0&#x27;) return res; &#x2F;&#x2F; Here&#x27;s the trick: res += (c == &#x27;s&#x27;) - (c == &#x27;p&#x27;); } } </code></pre> This gives a 3.7x speed compared to loop-1.c. The lower line count is also nice.
评论 #36623319 未加载
erualmost 2 years ago
Compare also <a href="https:&#x2F;&#x2F;codegolf.stackexchange.com&#x2F;a&#x2F;236630&#x2F;32575" rel="nofollow noreferrer">https:&#x2F;&#x2F;codegolf.stackexchange.com&#x2F;a&#x2F;236630&#x2F;32575</a> &quot;High throughput Fizz Buzz&quot; where someone uses assembly to generate Fizz Buzz at around 54-56GiB&#x2F;s.
gavinrayalmost 2 years ago
Fantastic post, I appreciated that the ASM was displayed in tabs as both &quot;standard&quot; and &quot;visual-arrows&quot;-annotated.<p>Kept me reading into the follow-up article.<p>Also, I love the UI of this blog.
评论 #36628757 未加载
arun-mani-jalmost 2 years ago
Any guide on how a person who uses Python or JavaScript can learn such things? I mean knowing which assembly code would be better, which algorithm makes better usage of processor etc.? :)<p>Also, how is such optimization carried out in a large scale software? Like, do you tweak the generated assembly code manually? (Sorry I&#x27;m a very very very beginner to low-level code)
评论 #36628470 未加载
评论 #36628441 未加载
评论 #36657790 未加载
评论 #36629830 未加载
vardumpalmost 2 years ago
I think it&#x27;s straightforward to optimize to a point it&#x27;s maybe about 10x faster than the &quot;optimized&quot; version. The answer is of course SIMD vectorization.
red2awnalmost 2 years ago
I experimented with different optimizations and ended with 128x speedup. The improvement mainly comes from manual SIMD intrinsics, but you can go a long way just by making the code more auto-vectorization friendly as some other comments have mentioned. See:<p><a href="https:&#x2F;&#x2F;ipthomas.com&#x2F;blog&#x2F;2023&#x2F;07&#x2F;n-times-faster-than-c-where-n-128&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;ipthomas.com&#x2F;blog&#x2F;2023&#x2F;07&#x2F;n-times-faster-than-c-wher...</a>
ammalmost 2 years ago
Back-of-the-envelope approach that should eliminate most branching:<p><pre><code> int table[256] = {0}; void init() { table[&#x27;s&#x27;] = 1; table[&#x27;p&#x27;] = -1; } int run_switches(char *input, int size) { int res = 0; while (size-- &gt;= 0) res += table[input[size]]; return res; }</code></pre>
评论 #36657666 未加载
lukas099almost 2 years ago
Would it be possible to write a code profiler and compiler that work together to optimize code based on real-world data? The profiler would output data that would feed back into the compiler, telling it which branches were selected most often, which would recompile optimizing for the profile. Would this even work? Has it already been done?
评论 #36624576 未加载
olliejalmost 2 years ago
I see other people have done minor rewrites, but the post does mention reordering branches, so the obvious question is whether there was any attempt to use PGO, which is an obvious first step in optimization.
einpoklumalmost 2 years ago
A very instructional post. I wish more people had such a level of mastery of GPU assembly and its effects, and would post such treatments on outsmarting NVIDIA&#x27;s (or AMD&#x27;s) optimizers.
failuseralmost 2 years ago
Having a full-blown predicate support is so nice to have, but it interferes with compact instruction encoding.<p>Such bloated ISA like x86 might actually handle predicate support, but who will try such a radical change?
评论 #36630396 未加载
sitkackalmost 2 years ago
This is such a wonderful post! Heavenly.
rajnathanialmost 2 years ago
Really interesting. The recent HN article on branchless binary search also covered cmov: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35737862">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35737862</a>
RobotToasteralmost 2 years ago
Was the C compiled with optimisation enabled?
评论 #36623500 未加载
kristianpaulalmost 2 years ago
How fast is forth compared to C these days?
评论 #36632217 未加载
throwaway14356almost 2 years ago
naive q: could one just count one of the letters and subtract it from the total number of letters?
评论 #36627211 未加载
orlpalmost 2 years ago
I made a variant that is (on my Apple m1 machine) 20x faster than the naive C version in the blog by branchlessly processing the string word-by-word:<p><pre><code> int run_switches(const char* input) { int res = 0; &#x2F;&#x2F; Align to word boundary. while ((uintptr_t) input % sizeof(size_t)) { char c = *input++; res += c == &#x27;s&#x27;; res -= c == &#x27;p&#x27;; if (c == 0) return res; } &#x2F;&#x2F; Process word-by-word. const size_t ONES = ((size_t) -1) &#x2F; 255; &#x2F;&#x2F; 0x...01010101 const size_t HIGH_BITS = ONES &lt;&lt; 7; &#x2F;&#x2F; 0x...80808080 const size_t SMASK = ONES * (size_t) &#x27;s&#x27;; &#x2F;&#x2F; 0x...73737373 const size_t PMASK = ONES * (size_t) &#x27;p&#x27;; &#x2F;&#x2F; 0x...70707070 size_t s_accum = 0; size_t p_accum = 0; int iters = 0; while (1) { &#x2F;&#x2F; Load word and check for zero byte. &#x2F;&#x2F; (w - ONES) &amp; ~w has the top bit set in each byte where that byte is zero. size_t w; memcpy(&amp;w, input, sizeof(size_t)); if ((w - ONES) &amp; ~w &amp; HIGH_BITS) break; input += sizeof(size_t); &#x2F;&#x2F; We reuse the same trick as before, but XORing with SMASK&#x2F;PMASK first to get &#x2F;&#x2F; exactly the high bits set where a byte is &#x27;s&#x27; or &#x27;p&#x27;. size_t s_high_bits = ((w ^ SMASK) - ONES) &amp; ~(w ^ SMASK) &amp; HIGH_BITS; size_t p_high_bits = ((w ^ PMASK) - ONES) &amp; ~(w ^ PMASK) &amp; HIGH_BITS; &#x2F;&#x2F; Shift down and accumulate. s_accum += s_high_bits &gt;&gt; 7; p_accum += p_high_bits &gt;&gt; 7; if (++iters &gt;= 255 &#x2F; sizeof(size_t)) { &#x2F;&#x2F; To prevent overflow in our byte-wise accumulators we must flush &#x2F;&#x2F; them every so often. We use a trick by noting that 2^8 = 1 (mod 255) &#x2F;&#x2F; and thus a + 2^8 b + 2^16 c + ... = a + b + c (mod 255). res += s_accum % 255; res -= p_accum % 255; iters = s_accum = p_accum = 0; } } res += s_accum % 255; res -= p_accum % 255; &#x2F;&#x2F; Process tail. while (1) { char c = *input++; res += c == &#x27;s&#x27;; res -= c == &#x27;p&#x27;; if (c == 0) break; } return res; } </code></pre> Fun fact: the above is still 1.6x slower (on my machine) than the naive two-pass algorithm that gets autovectorized by clang:<p><pre><code> int run_switches(const char* input) { size_t len = strlen(input); int res = 0; for (size_t i = 0; i &lt; len; ++i) { char c = input[i]; res += c == &#x27;s&#x27;; res -= c == &#x27;p&#x27;; } return res; }</code></pre>
评论 #36636128 未加载
评论 #36630130 未加载
评论 #36630390 未加载