I applaud the effort but it does not address the elephant in the room: openssl engines are synchronous. If you use openssl in an event-loop style server (coincidentally, OP's own web server h2o appears to be event-loop based), this means that the whole server blocks if the RSA operation blocks.<p>For a hardware accelerator that may not be so bad but if you are trying to separate the HSM via the network, to minimize what an attacker can do after compromising the httpd, then every packet loss or an outage or network delay (or, more generally, any latency) would block the whole httpd.<p>Cloudflare proclaimed a while ago that they had a way to do essentially this (with nginx+openssl) and they said their solution was non-blocking, but they did not publish the code as far as I can tell.<p>I think if one wanted to solve this problem properly, larger architectural changes to openssl would be necessary. Please correct me if I'm wrong!<p>EDIT: Also, if you move out the RSA operation, ideally you'd want to distribute the work over more than one CPU core. If the operation is synchronous, you can't really do that.