Painless NGINX Ingress

I think this blog post is one turn of the crank away from a truth we&#x27;re all about to learn: don&#x27;t hand roll your own Kubernetes ingress.<p>Dealing with the traffic handling between your users and your code is not a trivial problem. Like all good ops problems, you can fix it with good tools, deep knowledge of those tools, fine-grained observability, and smart people running all that.<p>This has been the recipe for a couple of really successful SaaS offerings. Individual servers? Datadog. CDN? Akamai &#x2F; Fastly.<p>Disclaimer: I work at one of those companies, Turbine Labs, and we&#x27;re trying to make ingress better. Here&#x27;s a presentation from our CEO on Kubernetes ingress, and why the specification creates the problems that this blog post is trying to fix. <a href="https:&#x2F;&#x2F;www.slideshare.net&#x2F;mobile&#x2F;MarkMcBride11&#x2F;beyond-ingresses-better-traffic-management-in-kubernetes" rel="nofollow">https:&#x2F;&#x2F;www.slideshare.net&#x2F;mobile&#x2F;MarkMcBride11&#x2F;beyond-ingre...</a>

This is a great read. I know the <i>single cluster for all env</i> is something that is sort of popular but it&#x27;s always made me uncomfortable for the reasons stated in the article but also for handling kube upgrades. I&#x27;d like to give upgrades a swing on a staging server ahead of time rather than go straight to prod or building out a cluster to test an upgrade on.<p>I tend to keep my staging and prod clusters <i>identical</i>, even names of services (no prod-web and stage-web, just web).<p>I&#x27;ll set them up in different AWS accounts to clearly separate them and the only difference they have is the DNS name of the cluster and who can access them.<p>Edit: I suck at italicizing and grammar.

Cool read. I don&#x27;t use Kubernetes but I learned a few things from this blog post that are applicable to my ECS environment.<p>The NGINX config part is tricky and it didn&#x27;t come to mind that many programs will try to be smart about machine resources and it won&#x27;t work in the container world as expected. This was a good reminder. OP didn&#x27;t mention what Linux distro he&#x27;s using and what are all of the OS-level configs he changed in the end of the day; I&#x27;d like to see that (was there any config not mentioned in the post?).<p>It&#x27;s awesome that OP had lots of monitoring to guide him through the problem discovery and experimentation. I need more of this in my ECS setup. I didn&#x27;t hop on the Prometheus train yet, by the way.

Shameless plug! The insights in this article are pretty deep but if you&#x27;re looking for just a clumsy step 1 to setting up the NGINX ingress controller on Kubernetes, check out what I wrote:<p><a href="https:&#x2F;&#x2F;vadosware.io&#x2F;post&#x2F;serving-http-applications-on-kubernetes&#x2F;" rel="nofollow">https:&#x2F;&#x2F;vadosware.io&#x2F;post&#x2F;serving-http-applications-on-kuber...</a><p>The most important thing that I found out while working on the NGINX controller was that you can just jump into it and do some debugging by poking around at the NGINX configuration that&#x27;s inside it. There&#x27;s no insight in there as deep as what&#x27;s in this article, but for those that are maybe new to Kubernetes, hope it&#x27;s helpful!

&quot;Most Linux distributions do not provide an optimal configuration for running high load web servers out-of-the-box; double-check the values for each kernel param via sysctl -a.&quot;<p>This is not true, if you run Debian &#x2F; CentOS7 &#x2F; Ubuntu, out of the box the settings are good. The thing you don&#x27;t want to do is start to modify the network stack by reading random blogs.

NGINX also has their own ingress controller (in addition to the kubernetes community version): <a href="https:&#x2F;&#x2F;github.com&#x2F;nginxinc&#x2F;kubernetes-ingress" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;nginxinc&#x2F;kubernetes-ingress</a>

Great read!<p>&gt;&gt; &quot;Let me start by saying that if you are not alerting on accept queue overflows, well, you should.&quot;<p>Does anyone know how to effectively keep a tab on this on a docker container running nginx open source? I have an external log&#x2F;metrics monitoring server that could alert on this, but I&#x27;m asking more on the lines of how to get this information to the monitoring server.

Am I correct in assuming that there is the Kube Service IP routing happening via iptables DNAT to get the request into the Kube running the Ingress Controller, and then the Ingress Controller is on top of that routing traffic to another Service IP which also has to go through the iptables DNAT?

Thanks for sharing your experience. I&#x27;ve got great insights to double check in my current environment.<p>Could you share which version of NGINX you found the issue with the reloads? Which version the fix was released?<p>PS.: I find it interesting&#x2F;brave that you use a single cluster for several environments.

Great read! Especially the part on ingress class which I didn&#x27;t know about. Would you mind sharing some of your grafana dashboards?

I still don&#x27;t really understand the benefit of an Ingress controller versus just a Service &gt; Nginx Deployment.

ingress+overlay network confusion was the reason why we moved from k8s to Docker Swarmkit.<p>I still keep hoping for kubernetes kompose (<a href="https:&#x2F;&#x2F;github.com&#x2F;kubernetes&#x2F;kompose" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;kubernetes&#x2F;kompose</a>) to bring the simplicity of Docker Swarmkit to k8s.<p>Or will Docker Infrakit bring creeping sophistication first and eat kuberentes lunch ? (<a href="https:&#x2F;&#x2F;github.com&#x2F;docker&#x2F;infrakit&#x2F;pull&#x2F;601" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;docker&#x2F;infrakit&#x2F;pull&#x2F;601</a>)

Why does everyone use reverse proxies? It seems complex and inefficient. Why not serve xhr&#x27;s and other dynamic content from the app server(s) and static content from a static webserver?