Deno vs. Bun performance is rigged

Im surprised performance at this level even matters to most folks. Like if you truly thought this microbenchmark was the reason to choose one runtime over another Id be shocked. It makes it equally surprising that this error from the Deno crew, who should all know better.<p>In either case, I hope they announce a correction and move on to more important matters. If youre trying to shave another tiny bit of rps out of your boxes then thats an incredible success problem; not the kind 99.999 of companies will need.

&gt; Deno is a multi-threaded server that utilizes in this test almost 2x the CPU-time while Bun is running single-threaded utilizing only 1x the CPU-time.<p>So we should intentionally handicap Deno? This is complete nonsense. If Bun wants to come out ahead of Deno, then they should also consider scaling with the number of CPU cores.<p>Single core CPUs are few and far between those days. Real-world performance is what matters.

This post doesn’t link to the original benchmark, or any code to verify what’s being suggested. Along with the click-bate heading, I think it’s fairly irrelevant.<p>My assumption is that this is a micro benchmark doing just an echo response, that is so far from what Bun&#x2F;Deno&#x2F;Node will ever be used for it’s just a farce.<p>The speed of your application frameworks http server will (almost) never be your bottleneck.<p>Developer experience, available libs, security model, packaging, deployment story, and to some extent resource usage (within reason) are far more important.

Maybe it&#x27;s just me, but there are some things I don&#x27;t understand about this article.<p>&gt; Deno is a multi-threaded server that utilizes in this test almost 2x the CPU-time while Bun is running single-threaded utilizing only 1x the CPU-time.<p>How do we know this? Also, when the author says that Deno is getting &quot;almost 2x the CPU-time&quot;, does that mean Deno was being run on a 2-core machine?<p>&gt; If we run the same test on a MacBook Air M1, with latest Bun v. latest Deno, we get the following:<p>What &quot;same test&quot; was run? The M1 MacBook Air has an 8-core CPU. If the &quot;same test&quot; means the test that the Deno people ran in the presentation, shouldn&#x27;t Deno still be beating Bun due to the multi-core advantage?<p>I&#x27;m very willing to believe everything in the article. But especially with the author requesting that we verify and think critically, it would be helpful to have more details.

Meanwhile the author does the same omission of important details that Deno is being criticized for: the M1 has 8 cores, yet there&#x27;s no mention about how those are used in the Deno vs. Bun benchmark. Did Deno only run on one core? Did they run 8 Bun instances in parallel? Who knows.

I think nobody should be surprised that two startups competing for more or less the same space end up fighting over benchmarks. My recommendation is to ignore all benchmarks altogether and simply run the tests yourself.

If you care about performance <i>that</i> much you should be using neither. What you are buying into with these is DX, including features and language portability with other parts of your stack. As long as they are the same overall performance class, it doesn&#x27;t matter.

Not rigged at all. Just a joke of a microbenchmark trying to equate a toy with an actual runtime.<p>Bun may be great someday, but right now it’s missing such basic functionality that any benchmark is a farce. Nobody cares that you can quickly ack a request a half million times a sec if you can’t do something as basic as spawn a child process.

It seems like there is some battle between the two with an article bashing Bun&#x27;s performance metrics by a Deno contributor just the other day. The article doesn&#x27;t do nearly as good of a job pointing out the differences though and the allegations by the Deno contributor which appear to be legitimate.<p>Also, just because Bun is single threaded doesn&#x27;t mean they should get special exceptions in performance testing. If they don&#x27;t like the numbers then give a comparable test with Bun using multithreading.

Micro and synthetic benchmarks are useful, however people tend to attribute WAY too much importance to them because it quickly becomes a battle of egos IMO.<p>It&#x27;s entertaining seeing these 2 camps throwing jelly at each other, it&#x27;s healthy to a point and when it stops being healthy by that time most have moved on to the new shinny thing.<p>I suspect since both have taken investor money, that is a factor too.

Post and heading are written to attribute this to malice, without offering any proof. It seems likely given how new Bun is that the benchmark writers simply lacked familiarity.

And the original benchmarks of bun vs deno had bun using a native-code http server against a deno using a js-code http server, afaik.<p>And the bun-specific-fork-of-react vs deno + normal react, was suspect too.

&gt; Please - verify, verify, verify and think critically about what you read.<p>If you&#x27;re going to excoriate someone for an improper benchmark, and then provide one of your own and advise your audience to &quot;verify,&quot; then it might be wise to include instructions for how to reproduce your results.

&quot;We interrupt the presentation of our project to bring you this important message about the existence of a similar project that we are afraid of, but think we beat on some benchmark of limited relevance. Having said that, please don&#x27;t look up that project, let alone use it! ... We now return you to the presentation of our project.&quot;

Why isn’t bun utilising multiple cores?

I&#x27;m surprised to read that &quot;Bun actually sits above a URL router capable of matching methods, URL wildcards and parameters&quot;.<p>I thought bun was just a JS runtime, though I guess thinking about it maybe I don&#x27;t even understand what a JS runtime <i>is</i>.<p>I always figured nodejs is v8 + an stdlib. I think bun is a custom zig-js engine + an extended stdlib (website claims &quot;batteries included&quot;)?<p>Isn&#x27;t the router generally &quot;application&#x2F;framework (i.e vue) specific&quot;?<p>Does stuff like Express or Nextjs not actually ship a webserver and just use the underlying &quot;runtime&quot;&#x27;s `http.serve` function? Is it possible to make an analogous &quot;webbrick to unicorn&quot; server swap while still running nodejs or are you swapping runtimes to do that?

Gotta mention <a href="https:&#x2F;&#x2F;www.techempower.com&#x2F;benchmarks&#x2F;#section=data-r21&amp;test=json" rel="nofollow">https:&#x2F;&#x2F;www.techempower.com&#x2F;benchmarks&#x2F;#section=data-r21&amp;tes...</a> is these threads always.

The most problematic thing about the state of the world for web framework benchmarks is that some of them drop a lot of real world requirements while showcasing high RPS.<p>E.g. a signficant number of frameworks on the techempower benchmark don&#x27;t run with any timeouts in the various stages of a HTTP request lifecycle. That would mean if one would deploy those in the real world the server would sooner or later run out of memory or fd&#x27;s due to broken client connections. Once you add timeouts, performance would already drop.<p>Then there is logging and metrics, which are absolutely required for any production setup, but never included in any of those proof of concept setups.<p>If the performance after those mandatory things is added drops 2 to 4x, the conclusions from the benchmarks can be very different.

Related flagged article with good context&#x2F;comments that a lot of HN readers probably didn&#x27;t see because it got flagged:<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=33199351" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=33199351</a>

I&#x27;m curious if the Node.js setup that resulted in these numbers was also the default single process, rather than the cluster setup that&#x27;s included with Node, but not configured by default.

I&#x27;ve been a little puzzled about the latest runtime competition. Bun is a great package manager but I don&#x27;t see why you&#x27;d choose it as a runtime. Ditto with Deno and whatever other runtimes have entered the competition. Seems like you&#x27;d start using one, realize npm compatibility is rather poor, and pick the runtime that actually lets you, yknow, do stuff, i.e. node.

Reminds me of how each version of Windows is &quot;the fastest Windows ever!&quot; while the minimum system requirements increase with each version of Windows.<p>However <i>do</i> the geniuses at Microsoft manage to make Windows run faster while only requiring faster hardware?