Posted this yesterday on Reddit and gathered some good information but curious what the folks of Hacker News would add to the list I put together on the three main reasons as to why adopting Kubernetes for organizations is challenging.<p>1. Provisioning and managing Kubernetes clusters through their lifecycle is difficult and grows in complexity as you scale. Managing Kubernetes upgrades, with API version deprecations or other small behavioral changes causing outages or lost cycles (Take Reddit for example)<p>2. Maintaining a Kubernetes YAML codebase for application development is error-prone and frequently untestable.<p>3. Existing tooling is powerful, but lacks enterprise-grade features. Tools do not integrate well with each other and still require manual integration with the rest of the Kubernetes stack.
1. Provisioning and scaling clusters is inherently difficult regardless of tools used.<p>Unless your cloud provider does this for you. This is one of the first challenges. The problem of providing a tool that makes this simpler is that the biggest pain of k8s provisioning is choice explosion. A problem many others have already tried solving by creating distributions, which there now are so many of that you have a choice explosion of distributions instead, with no clear understanding of compatibility or what this black box does. What need to be done is remove choices from the core and streamline installation as one standard package, not adding even more abstractions and mystery on top.<p>2. Once you have a cluster running, there are less excuses. Any alternative IaC is equally untestable. In fact k8s is one of the more testable options since you can run a local cluster with Kind and the yamls don’t suffer from configuration drift.<p>Yaml and string templated Helm is hell, anyone who can come up with a more type safe language would get a gold medal. HCL for k8s would be fantastic. Staying away from Helm entirely also reduces a lot of accidental complexity, basic manifests get you very far.<p>3. Same applies to other tools.
If you're struggling to get prometheus, nginx, postgres and other battle-tested tooling working in k8s, you're probably also not doing it well in docker-compose, on bare metal or anywhere else because it's the same darn software with the same damn options. No matter what you choose you're going to be creating system services to manage the life cycle of an application and exposing a port or ip address - there is no escape
Kubernetes is overkill to my org which only manages about 15 production servers running monolithic software. We don't have scaling issues or performance bottlenecks. Everything is very simple to manage and update. Compared to Kubernetes, i'd be adding additional infrastructure just for kubernetes itself or i'd have to pay more for a cloud to do this for me, just to put our own software on top of this layer that we have no use case for besides learning the new shiny tech.
Kubernetes has always been overkill for most people, it’s for very large scale enterprises. Not for your typical CRUD app.<p>I don’t know what it is, but so many is focused towards being infinitly scalable without thinking about the complexity that grows with it.