Had to look by chance at that folder and found out the full history of certificates and keys were sitting there. I don't know what to think, isn't this like having a record of all your previous passwords?, but worse.
It’s not nearly as bad as you think. If you’re a Let’s Encrypt user, there’s a high likelihood that your servers (and your clients) were negotiating PFS cipher suites.<p>If they were, a compromise of the private key wouldn’t mean that you could break past intercepted communications.<p><a href="https://en.m.wikipedia.org/wiki/Forward_secrecy" rel="nofollow">https://en.m.wikipedia.org/wiki/Forward_secrecy</a>