Raspberry Pi Homelab with Kubernetes

I&#x27;ve found k3s (<a href="https:&#x2F;&#x2F;k3s.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;k3s.io&#x2F;</a>) to be extremely easy to use for setting up and running kubernetes on a group of Raspberry Pis. Specifically, I followed this guide to get my clusters up and running quickly and it&#x27;s worked out pretty well: <a href="https:&#x2F;&#x2F;blog.alexellis.io&#x2F;test-drive-k3s-on-raspberry-pi&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.alexellis.io&#x2F;test-drive-k3s-on-raspberry-pi&#x2F;</a>

I understand the allure of real hardware and doing this on a physical cluster but I&#x27;m always surprised not to see K8s&#x2F;k3s&#x2F;whatever running under (inside?) LXD as a learning&#x2F;experimentation tool discussed more often.<p>The physical cabling, underlying operating system, bootstrapping etc strikes me as the least of the K8s learning experience and the only advantage of a toy running on Pis vs LXD. Not to mention most K8s deployments these days will be in some cloud provider of choice where most of this is handled... In that case LXD on a local machine with software network bridging, etc probably more closely approximates what most people will go to production with anyway.<p>I&#x27;m a little sour on the whole &quot;LXD only officially distributed via snap&quot; thing too but at least on Ubuntu 18.04 and forward getting a toy X node Kubernetes cluster up and running is trivial and costs nothing more than RAM and disk space. As is commonly known LXD doesn&#x27;t even require a hypervisor so the hardware requirements are (essentially) anything x86_64. It&#x27;s also fun to spin up&#x2F;destroy any number of instances at will just using the command line.<p>MicroK8s even provides an LXD how-to that should work for your flavor of choice (with a little adaptation, of course):<p><a href="https:&#x2F;&#x2F;microk8s.io&#x2F;docs&#x2F;lxd" rel="nofollow">https:&#x2F;&#x2F;microk8s.io&#x2F;docs&#x2F;lxd</a><p>Of course if you&#x27;re actually doing some kind of edge deployment or whatever on actual Raspberry Pi&#x2F;armv7&#x2F;arm64 hardware that&#x27;s the obvious way to go (or you can just run LXD on your Pi) :).

As has become my habit lately, I&#x27;ll chime in and say if you&#x27;re interested in accessing your self-hosted services from the internet, IMO tunneling is the way to go: <a href="https:&#x2F;&#x2F;github.com&#x2F;anderspitman&#x2F;awesome-tunneling" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;anderspitman&#x2F;awesome-tunneling</a>

I&#x27;ve been looking into doing an SBC K8s cluster, but had my eyes on the ODroid N2+. Its big.LITTLE design with 4 high-performance cores and 2 low-performance ones would not only boost the compute capacity vs the RPi4, but also perhaps cgroups on the nodes could be configured such that pods only run on the 4 high-performance cores, leaving the 2 low-performance ones available for system daemons, kubelet, etc. The biggest drawback with the N2 is that it maxes out at 4GB of RAM, which might not be sufficient for the cluster master nodes. So maybe the master nodes could be 8GB RPi4s, while all the worker nodes are N2s.

I have what I feel is an irrational desire to build an RPi k8s cluster. I feel like I would be better served by a small x86 box, but something about having a real cluster really appeals to me.

For those like me who are a great deal fainter of heart than the author, there&#x27;s always k3s [1] (installed e.g. via k3sup [2]) to take care of the &quot;hard&quot; Kubernetes part, at least until something breaks, though I&#x27;ve been lucky so far. Will run great on a Pi 4 and decently well on a 3&#x2F;3+. The only big issue I&#x27;ve had so far was finding 32-bit armv7 images (I&#x27;m running Raspbian; I hear Ubuntu can do 64bit just fine), most projects only ever publish arm64 ones.<p>[1] <a href="https:&#x2F;&#x2F;k3s.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;k3s.io&#x2F;</a><p>[2] <a href="https:&#x2F;&#x2F;github.com&#x2F;alexellis&#x2F;k3sup" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;alexellis&#x2F;k3sup</a>

I spent some time getting pi-hole to run on a raspi k3s cluster last year and wrote about it. Hopefully there&#x27;s something useful in my investigation for your current project.<p><a href="https:&#x2F;&#x2F;medium.com&#x2F;@subtlepseudonym&#x2F;pi-hole-on-kubernetes-87fc8cdeeb2e" rel="nofollow">https:&#x2F;&#x2F;medium.com&#x2F;@subtlepseudonym&#x2F;pi-hole-on-kubernetes-87...</a>

This person sets his host names to Foundation Trilogy planet names. This is good stuff.

Once upon a time the cool thing to do was to build RAIDs out of USB sticks. Or SD cards. Or floppy disk drives.<p>Raspberry Pi isn&#x27;t as laughable as that, but the fact is that what you are getting out of this is a learning opportunity, not a reliable or high-capacity computing platform.<p>And even there, 4 Pi4Bs with 8GB will cost you $400 and more, once you add in power supplies and SD cards and desiderata. The first hit on Craigslist just now was a $200 4 core i7 with 8GB of RAM (upgradable) and a 256GB SSD in a nice small case. Two of those will get you better overall performance and leave room for expansion.

Why are people still posting these Pi homelabs they have been done to death.<p>I run microk8s on KVM running in Mesos with openvswitch and Mikrotik CRS&#x2F;CCRs.

Awesome. Gonna give this a go. Recently acquired a 3rd Rasp4 to have a quorum of fast-ish ones.<p>Concerned about the ARM aspect though. On docker at least the ecosystem felt a lot smaller on ARM.

&quot;so that you have something pretty to show your non-technically inclined significant other as the output of your hard work&quot;<p>Gosh, that resonates... :)<p>Also very excited to read part 2!

I am going to print that xkcd and pin it above my desk.<p>Thanks for these articles. I once again realized I should really not do this (for now)