I haven't used kube in production yet.<p>However, I'm using mesos, marathon and chronos to manage a production environment with service discovery glue based on Route53.<p>Using Docker to ship an application to a well configured environment is just a delight, the amount of configuration needed is absolutely minimal.<p>However, I think people need to realize that it's "easy" if your services are not talking to each other and dependent on one another in a way. If service X is using service Y directly (via HTTP), it gets a bit more challenging.<p>The way I like to configure micro-service is based on messaging so you send a message to a queue and multiple satellite services can consume that message and do stuff with it.<p>If your services are dependent on one another, the configuration gets trickier and the maintenance gets a bit harder.<p>Good job by Box also contributing back to the core of Kube based on what they needed, based on it getting merged I am guessing other people will find it useful as well.
Anyone working on K8s at Box or I guess anywhere else that has deployed it partially feel free to answer this, but:<p>How do you handle gatewaying traffic into Kubernetes from non-K8s services? I've been trying to get a basic cluster out the door with one of our most stateless services, but I'm having a having a hard time just getting the traffic into it.<p>The mechanism I'm using is having a dedicated K8s nodes that don't run pods hold onto a floating IP to act as gateway routers into k8s. They run kube-proxy and flannel so they can get to the rest of things, but ksoftirqd processes are maxing CPU cores on relatively recent CPUs trying to handle about 2Gbps of traffic (2Mpps) which is a bit below the traffic level the non-k8s version of the service is handling. netfilter runs in softirq context, so I figure that's where the problem is.<p>Are you using Calico+BGP to get routes out to the other hosts? What about kube-proxy?
This is the 1st use case I have seen where microservices are starting to make sense.<p>My question is what about network security ? How is that part managed ?
Totally lost it at "e knew we'd ultimately need dozens (even hundreds) of microservices to be successful" and did not read any further. I am having a very hard time seeing that as a criteria for success, not to even mention imagining how that mess is managed. Is this really common to have so many microservices?