This is a great read. It's a fairly coherent summary of the things I intuitively felt great about Kubernetes, and discusses it plainly and explicitly.<p>Towards the end of the article, the authors mention the open challenges.<p>Unbeknownst to me, I had been working on something that attempts to address some of that. It is my sixth attempt at a tool like this. It came out from when I saw how many dev environments attempting to use Docker Compose ended up with a large, ad-hoc collection of Bash scripts.<p>The project is called Matsuri, and it is found here: <a href="https://github.com/shopappsio/matsuri" rel="nofollow">https://github.com/shopappsio/matsuri</a><p>It's intended as a framework for programmatically generating manifests and executing kubectl commands. It doesn't try to have a lot of opinions (at least, opinions that you can't change). The idea is that your platform support tool is like any other app, and should be tailored to your specific collection of apps.<p>It also has a notion of Apps, which are a bundle of K8S resources declared as dependencies. But that's as far as I gotten -- it has a fairly anemic convergence tool. I was more concerned about standardizing builds, pushes, updates (collection of rollouts, migrations, etc.), shell commands, "console" commands, etc. These are all achieved by expecting Apps to define callback hooks for those actions.<p>I don't know if this is useful or interesting to anyone else. But if anyone is interested, feel free to contact me about it.
It is interesting to see the evolution of cluster scheduling at Google. Mostly, it has been driven by the desire to improve resource utilization - with a second thought given to usability.
Their first go with Borg was centralized, correct, and enabled global state to make good scheduling decisions. Then they tried Omega, which never made it to production. But they subsequently separated scheduling from resource tracking in Borg (one scheduler node, ~5 paxos nodes handling resource notifications/communicating scheduling decisions with workers). Again, all optimizing utilization.
Kubernetes, however, is about making it easier for developers. So, one IP per pod - unlike Borg, where containers share the host IP. I like Kubernetes, but adoption has been slow. It will be interesting to see how it develops - whether it will subsume Mesos. How it will stack up against Hadoop/YARN.
The thing I'm still struggling to wrap my head around with kubernetes (and docker in general) is the management of data persistence.<p>If I have a pod of containers everything I read seems to say that containers need to be 'stateless' and if you're running kubernetes, will likely also be transient based on the system load & scale.<p>So if any container in a pod could go away or be spun up at any time, if it could live on any virtual machine in a cluster of physical machines...<p>where do you keep the physical database file so that it is accessible to all different instances? How do the multiple instances access it at the same time? Generally, how does the database layer "work" in container land?<p>It's the only piece that I really struggle to understand.
I always admire papers that make the virtual things we wrangle with all day seem even more physical.<p>This achieves that with its (afaik) novel usage of "hermetic":<p><pre><code> The key to making this abstraction work is having a hermetic container image.
</code></pre>
Poetry.