Most of the gnarly things I've had to deal with while building and operations K8s clusters and related infrastructure is around stateful apps.<p>Most modern cloud applications and services rely on a set of services for storage that are mostly hosted by a cloud provider. I understand Kubernetes' intention is to help support all workloads. My thought is that those starting now don't need to inherit all that complexity - code paths, operations, possible risks in terms of security and stability as that part of the code changes, and the myriad of configuration options.<p>Instead a stateless container orchestrator can focus on things that maybe more relevant to modern cloud apps - like improved autoscaling, traffic routing, placement, etc.<p>I'd be interested to learn about new applications that are being built that require persistent disk storage.
Yeah that's something I've thought about. Although K8s sells itself as a platform for building cloud native services, K8s itself is not cloud native, in the sense that doesn't take advantage of modern public clouds that much by default. Examples:<p>- Instead of having a dedicated control plane, something like DynamoDB could be used to store state.<p>- Clustering based on stuff like Tags (Hashicorp Consul does that).<p>- Suff like ExternalDNS and External Secrets Operator could be defaults.<p>- Every cloud platform has something similar to security groups and load balancers, so networking could be way simpler.<p>Prob many other ways of simplifying stuff.<p>The platform itself would just provide an API for accessing cloud services in a simple, common way.<p>Microsoft has a side car based project, totally forgot the name, that partially does that. It exposes http endpoints like localhost/secrets or localhost/invoke/$APP to all services, while using cloud services behind the curtain.<p>Edit: the project is called Dapr <a href="https://dapr.io/" rel="nofollow">https://dapr.io/</a>
k8s is the best solution for _universally_ bringing legacy pre-container "legacy" application patterns into a cloud. K8s won the battle (vs mesos, openshift) for a number of a reasons - but one of those reasons was absolutely not simplicity, rather k8s was better at handling the edge cases. This is a popular opinion held by Victor Farsick @ DevOps Paradox podcast and I happen to agree with him.<p>You are absolutely correct that k8s is not necessary for cloud-native environments. Existing comapnies usually have legacy applications, so they can't go "cloud native".<p>Once a company has started down the k8s path the org (by necessity) will start to specialize a k8s control layer and that will become embedded and you risk career-icide & being labelled a heretic if you try to challenge the rationality of using k8s. K8s creates a nice way sysops to keep devs in little boxes and limit the size of the craters they can make. All everybody wants at the end of the day is not to be hassled and not need to keep learning something new each day.<p>Dev's are usually horrible system operators since they fundamentally want to use code to solve problems rather than look for solutions others have built. K8s in this respect creates a very non-creative line of demarcation for system responsibilities. Devs should not be writing backup & logging applications.<p>If you're building a startup and you don't want the k8s complexity what you are suggesting is fine - but if you're going to work in big b0rg org, it's better to get in the k8s bandwagon - what you are saying will get you labelled a heretic, because the k8s admins have enough struggle keeping up with k8s complexity & beating devs into submission, the idea of learning anything else is unpopular.<p>It's straightforward to hire people with k8s expertise, but now this is a domain of knowledge that humanity has cultivated.<p>I remain of the unpopular opinion that k8s is really only suitable for companies which are the size of google (imho), the idea of having a massively complex & specialized administration layer for most companies is absolutely stupid. However the idea of refactoring legacy applications to be cloud native will never fly, too many risks & unknowns, disruption, big b0rg orgs dont want none of that.<p>So k8s is also a safe choice, it's not especially creative, it's burdensome but big b0rg orgs don't really care about any of that - these folks are unfortuantely the decision makers, and they don't understand the topics they are making decisions on, so popularity of tech, groupthink etc. are going to win there.