> There is one way to make the LLVM JIT compiler more usable, but I fear it’s going to take years to be implemented: being able to cache and reuse compiled queries.<p>Actually, it's implemented in LLVM for years :) <a href="https://github.com/llvm/llvm-project/commit/a98546ebcd2a692e0634c5b1a7e77471316ab6e0">https://github.com/llvm/llvm-project/commit/a98546ebcd2a692e...</a>
I'm still surprised there isn't a query/plan cache for PostgreSQL. I could easily see these two approaches working in harmony once it does, as frequent queries could end up being cached and more aggressively optimized with a cache to offset the compilation cost. Of course that adds a whole new layer of complexity and trouble.
Back in 386 era, there was a concept of self-modifying code (assembly). A similar like stencils presented here, but because code was a singleton, rarely a copy was made.<p>E.g. Doom on DOS used this optimisation techique, because otherwise you could not cram out enough performance from tight rendering loops on old CPUs.