Ahh, but PyPy is not being written by web developers -- or for them. The main guy behind the project is a research computer scientist (<a href="http://cfbolz.de/" rel="nofollow">http://cfbolz.de/</a>) and most of the people who are excited about it are also doing scientific computing.<p>I am one of them, and the ability to write scientific code in Python that runs as fast (or faster!) than C would be so awesome that I can hardly put it into words. Unlike web development, performance is still a huge bottleneck in this field; people still mill about for 3 weeks waiting for their job to finish just like it was the 1970s. Consequently, most code written for research projects consists of lot of arcane, difficult-to-read C, C++, or even (gasp!) FORTRAN.<p>The benefits of switching to Python in terms of development time, ease of debugging, reusability, potential for collaboration, reproducibility, etc. are immense, but the performance penalty is such that it rarely happens. Even in spite of that, Python already has a large academic following; if PyPy manages to pull off its stated goals, it would be revolutionary.
"What do I want? [...] Shared objects across [micro-]processes with copy-on-write; then you can efficiently share objects (like modules!) across concurrent processes without the danger of shared state, but without the overhead of copying everything you want to share."<p>Is overhead of modules really an issue on servers with GB of RAM? More importantly, almost every multi-threaded string library (like C++ STL) has moved <i>away</i> from sharing across threads and COW because the cost of the atomic operations is too high. See Herb Sutter's "Optimizations That Aren't (In a Multithreaded World)."<p><a href="http://www.gotw.ca/publications/optimizations.htm" rel="nofollow">http://www.gotw.ca/publications/optimizations.htm</a><p>Also, to say that speed is "really uninteresting" but latency is important seems like a contradiction. Latency is absolutely gated by speed for non-parallelizable problems. And even if the problem can be parallelized, speed is a <i>much</i> easier and simpler way to decrease latency than parallelizing.<p>I guess this reads to me more like a list of theoretically interesting ideas than a set of features that will actually help anyone in the real world.
For what it's worth, another 2c from another web developer - for me memory isolation and COW for module dependencies would be less useful than a single process and a lack of GIL (hold the flamethrowers for one second please) - allowing me to share on a multicore server resources like those same modules, as well as database connections, as well as any significant static data i may read in at start up, decreasing my operational complexity, etc. In a web server environment shared memory isn't particularly scary, to me anyway. Am I misunderstanding the reason for COW?<p>Also, the most important characteristic to me is latency. I don't care about throughput that much as long as it's "good enough", hardware is getting really cheap really quickly anyway (in terms of cores and RAM/$). Latency on the other hand is tough. While IO is normally going to be the biggest factor here, if the runtime allows me to consistently shave 5ms off of every request - that's still a great improvement. In this sense i would say that raw speed IS important to me.<p>Perhaps I haven't encountered the use cases Ian is talking about, it was difficult to figure them out from the blog post (i'm a relative novice in Python).<p>But I am hoping PyPy can at some point help with the concerns I have above, even lower latency by itself may be compelling enough to switch.
It seems like he could get most of what he wants without a new runtime. Diesel (<a href="http://dieselweb.org" rel="nofollow">http://dieselweb.org</a>) provides async without callbacks or Deferreds. Diesel1 used Python 2.5's yield expression, and Diesel2 switched to using greenlets. As long as your modules and your applications don't rely on global state (which they shouldn't anyway), you essentially get everything on this list. For web programming, cooperative multitasking usually works perfectly well and no state is generally shared between any 2 http requests, so I'm not sure how much a web programmer stands to gain from preemptive multitasking and completely separate address spaces with CoW.<p>The one thing Diesel is missing is good runtime introspection and more primitives to encourage reliability (a la Erlang), but if you hang out in #diesel you'll see that jamwt has already implemented a lot of that at Bump and he plans on pushing it down into the framework soon (probably for Diesel3).
The first two points are describing fork(), which python already has on unix-like systems. If he means something different, then he needs to explain more how exactly it would work (especially regarding hardware/os support).
I agree. I think there needs to be an effort to push towards concurrency in Pypy. They were optimistic that the GIL would be easier to factor out in their runtime, and I think that could be a great start. I'm all for new methods for concurrency, but I feel being able to make efficient use of native threads would bring a lot of momentum to the project. Even though it's not as big a deal as most people make it out to be, Python gets a bad rap for the well publicized GIL problems.
In addition to speed and the points raised by the OP, I am interested in static/duck typing as it relates to both speed and static error-checking and documentation. i.e., making debugging easier and faster. I somehow think this can happen while keeping almost all of Python's strengths and ease of development.