Analyzing Data 170,000x Faster with Python

For context, this is in response to &#x27;Analyzing Data 180,000x Faster with Rust&#x27; [0], previously discussed on HN [1]<p>[0] <a href="https:&#x2F;&#x2F;willcrichton.net&#x2F;notes&#x2F;k-corrset&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;willcrichton.net&#x2F;notes&#x2F;k-corrset&#x2F;</a><p>[1] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=37964161">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=37964161</a>

I’m all for not prematurely optimising and the point of the authors post is still valid, but it is hard to argue that the resulting code is more readable or understandable than the rust code.<p>The rust code is pretty simple, and plain, and doesn’t need any of these tricks

Interesting for sure, but that&#x27;s definitely a more painful optimisation journey than the Rust one, which only involved Rust. No custom JITing of a subset of Rust, or calling out to C.<p>I think the real takeaway from my experience of Rust vs Python performance is that in practice people don&#x27;t optimise 99% of Python so the real number of significance is the 8x speedup he got from just switching to Rust with no profiling whatsoever.<p>That&#x27;s actually quite a low difference compared to my experience - I&#x27;d say 50x is more typical.

Would be very interested in seeing how torch.compile() fares with OP&#x27;s numpy portions: <a href="https:&#x2F;&#x2F;pytorch.org&#x2F;blog&#x2F;compiling-numpy-code&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;pytorch.org&#x2F;blog&#x2F;compiling-numpy-code&#x2F;</a>

This seems like a meaningless claim because the core parts of both itertools and pandas libraries used here are written in C, unless I&#x27;m missing some context here.