> Benchmarking isn’t a perfect measure of how effective this change is (benchmarking is arguably never perfect, but that’s a different conversation), as the results would be too noisy to observe in a large project.<p>So... does that not mean that any micro-improvement would also be too small to be significant on any large project?
I remain skeptical of JRuby. As of about 2-3 years ago, there were many benchmarks touting it's performance, but in our large publicly deployed web app, it always underperformed and switching to MRI on Phusion more than doubled our performance on half the servers. We also spent like a developer-year on issues that accompanied being off the beaten path with JRuby, like various nokogiri complications.<p>I even contacted the devs at one point with a benchmark that ran jruby directly on strings for several minutes. The answer I got was always that it would eventually perform better than MRI with more warmup. Spoiler: it didn't. Well, maybe I was supposed to run it for another month and it would have.
(btw I am the author of this post, thread questions if wanted. Trying to avoid draining myself addressing all the comments. Take care in these strange times <3)
>'TruffleRuby has high potential in speed, as it is nine times faster than CRuby on optcarrot, a NES emulator benchmark developed by the Ruby Core Team.'<p>Really? But now the latest results are in apparently and judging by these benchmarks[0], I don't know what to make of this claim given how slow TruffleRuby is when compared to other similar languages or even Ruby. But its clear that its LLVM-based cousin Crystal still blows it out of the ocean in every benchmark if you're really talking about 'high performance'...<p>[0] <a href="https://github.com/kostya/benchmarks" rel="nofollow">https://github.com/kostya/benchmarks</a>