I have a slightly weird and probably completely unworkable "idea" that has nothing to do with this particular GIL improvement but with the broader GIL issue.<p>The problem with removing the GIL seems to be garbage collection. I wonder why it's not possible to introduce a new type of object reference that exempts referenced objects from being garbage collected.<p>Then multiple Python interpreters could be started in separate threads, each with its own unmodified GIL, and the only thing they could access would be their own thread local data and those special shared objects.<p>What this amounts to is basically an implementation of a multi process architecture on top of a multi threaded architecture. The crucial difference is that the memory shared among the interpreters could hold pointers and thus proper in memory data structures and not just a BLOB into which everything has to be serialized as in the case of conventional shared memory.<p>Of course the shared objects would have to be manually deleted.<p>One issue I see is that when such a special object is created, all the objects it creates recursively would also have to be allocated in that pool of special objects. But I think it should be possible to use some kind of global flag to special case the allocator.<p>Well, there is probably a huge number of issues with this kind of trickery and it's definately not a long term solution. But I'd love to use Python much more than I do and the GIL issue is what prevents that for me at the moment.
Knowing little of Python, is it the case that the GIL was not initially deemed a bad scheme because multi-processor x86 machines were a rarity in the mid-90's (when Python was young)? I presume that Sun desired Java to be multiprocessor friendly because it had multi-cpu SPARC machines at the time of Java's design?<p>And, for those using Python for high-volume production systems, how do you cope? Do you run one process (say a Zope instance) per core and load balance?
"""Does it work? Yes it's better: Sequential 23.5 seconds. Threaded 24.0 seconds."""<p>How is that better? It is an improvement over earlier versions of Python. Better would be if the threaded version would actually be twice as fast.