Why is Kubernetes adoption so hard?

It&#x27;s a marketing post, but since you and me are here already:<p>Replace YAML with proper &quot;general purpose&quot; featureful configuration language like Nix (or a typed version of it: Nickel). You could do Dhall or Jsonnet, but IMO, Nix is the way to go and keeps gaining traction.<p>Abandon Dockerfiles, and let the containers be described in the same general purpose language (see <a href="https:&#x2F;&#x2F;nixos.wiki&#x2F;wiki&#x2F;NixOS_Containers" rel="nofollow">https:&#x2F;&#x2F;nixos.wiki&#x2F;wiki&#x2F;NixOS_Containers</a> ), that the cluster can build and handle the same way.<p>Have the cluster itself be described in the same configuration language.<p>Wrap it all the proper document library APIs, so the most common use-cases, are a handful of code, with ability to expand to lower level details once it needed. Hey, did I mention that we have a general purpose configuration language that makes it possible?<p>At the end it should be possible to `git init .`, write 10 lines of Nix code (import stdlib, import 1 node standard cluster, run a servive with hello-world application), call `deploy-this` with your cloud access key and get a k8s cluster with a hello world application behind a TLS.<p>Then when you decide you need more nodes, more services, you just add new lines, or rewrite existing high level ones into more detailed calls.<p>All of this currently required gluing tons of things together (Docker, Terraform, kops, Helm, Kustomize, ArgoCD and what not) all using different programming languages, xmls, jsons, yamls, HCLs, Dockerfiles and so and so.

I am going to state a somewhat unpopular opinion, Kubernetes in their current form are just short from unusable in complex production environment. In order to run a full Kubernetes stack, you need to get a multitude of components running, each with own name with unknown effects on the underlying system.<p>I have experiences with two forms of deployment: one was an on-prem installation two years ago, on which one of the 13 Java 8 applications had very large latencies when accessing Oracle DB, otherwise working fine when it was deployed on simple VM. All of those applications had been done with the same DB logic, and we couldn&#x27;t find the issue on our own, so we asked third-party to debug this issue for us: they couldn&#x27;t pin point the problem, even with commercial tools. Their answer was just, something is off with your Kubernetes installation and that was it.<p>My second, on-going experience, is my current assignment with Fortune 500 company, that uses GKE for running hundreds of nodes, after migrating from on-prem VMs. Almost every other week (99% reliability - yeah right), some part of the system just dies and leaves services unreachable or unresponsive. There is a continuous effort to solve this issues and even Google support was contacted with the answer boiling down to: shit happens, deal with it. The only solution in those situations is either to have alarms go off so that Ops can restart something, or just to wait until everything comes back up again on its own.<p>The whole ecosystem was a good idea that lacks proper tool and stability to provide substantial benefits over the bunch of VMs, IMO.

Because helm charts lack a schema. Programming by yaml configuration means you spend time searching github for an example kind of like what you&#x27;re trying to do instead of having actual tools which allow auto-completion.<p>Editing helm charts it the ultimate form of throwing spaghetti at the wall and praying it works, without being able to actually test anything locally.

The Kubernetes API allows you to do a hell of a lot, and people can go overboard with that (certainly I have).<p>Inner peace arises from looking at all of the possibility and limiting yourself to what you actually need.

This is a pure marketing high bullshit article. This is akin to spam.

One of the big problems I&#x27;m having in building a bare metal cluster is the awful error messages.<p>The flexibility of allowing many different CRIs, CNIs and storage providers does let you find something that suits your use-case, but that generality also means the error messages can&#x27;t be as specific as a fully integrated solution.<p>The issue is exacerbated when the OS and cluster are hardened to CIS benchmarks with policy managers - trying to figure out what is preventing your app from running and why is just so much harder than it needs to be.

Without following the link I would say <i>because YAML</i>.

K8s and the like will never replace coding for actual computers nor do they replace the many layers they sit upon. Networking isn’t well thought out, and we generally need to stop propagating the idea that this is the future of application execution, it’s a hack, a clever hack, but insufficient to obviate hypervisors and operating systems. If you truly seek shrinkwrapped application s look into exokernels.

It feels like the article was written with the audience of business people, not ops. I asked ChatGPT what Cluster API is or what it tries to solve and it made much more sense.<p>Cluster API does not have an integration with Karpenter and there are more limitations [1], rendering it unusable in my use case.<p>[1] <a href="https:&#x2F;&#x2F;cluster-api-aws.sigs.k8s.io&#x2F;roadmap.html" rel="nofollow">https:&#x2F;&#x2F;cluster-api-aws.sigs.k8s.io&#x2F;roadmap.html</a>

Beware - marketing post.