TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Running Nomad for a Home Server

278 点作者 elliebike大约 4 年前

25 条评论

tetha大约 4 年前
We&#x27;re pulling the trigger tomorrow to migrate the first productive system to nomad, and launch 1-2 new products on nomad first. It&#x27;s quite exciting.<p>We chose nomad there, because it&#x27;s a business requirement to be able to self-host from an empty building due to the data we process - that&#x27;s scary with K8. And K8 is essentially too much for the company. It&#x27;s like 8 steps into the future from where most teams in product development and operations are. Some teams haven&#x27;t even arrived at the problems ansible solves, disregard K8.<p>The hashicorp stack with Consul&#x2F;Nomad&#x2F;Vault&#x2F;Terraform allows us to split this into 4-8 smaller changes and iterations, which allows everyone to adjust and stay on board (though each change is still massive for some teams). This results in buy-in even from the most skeptical operations team, who are now rolling out vault, because it&#x27;s secure and solves their problems.<p>Something that overall really impressed me: One of our development teams has a PoC to use nomad to schedule and run a 20 year old application with windows domain requirements and C++ bindings to the COM API. Sure, it&#x27;s not pretty, it&#x27;s not ready to fail over, nomad mostly copies the binaries and starts the system on a prepared, domain joined windows host... but still, that&#x27;s impressive. And it brings down minor update times from days to weeks down to minutes.<p>Being able to handle that work horse on one hand, and flexibly handling container deployments for our new systems in the very same orchestration is highly impressive.
评论 #26150018 未加载
评论 #26149203 未加载
marvinblum大约 4 年前
I did&#x2F;do run both myself, Kubernetes and Nomad, and it was a million times easier to set up Nomad (including Consul) on bare metal than it was to set up Kubernetes. Kubernetes offers more features, but you most likely don&#x27;t need them and the increase in complexity makes it a pain to maintain. I&#x27;m running a three-node cluster on Hetzner [0] for Pirsch [1] right now and haven&#x27;t had any difficulties whatsoever when upgrading and modifing the cluster. Adding Vault is a bit tricky, but I&#x27;m sure I&#x27;ll get that done too. I highly recommend trying it if you think about maintaining your own infrastructure!<p>Let me know if you have any questions. I plan to write an article&#x2F;kind-of-tutorial about the setup.<p>[0] <a href="https:&#x2F;&#x2F;www.hetzner.com&#x2F;de&#x2F;cloud" rel="nofollow">https:&#x2F;&#x2F;www.hetzner.com&#x2F;de&#x2F;cloud</a><p>[1] <a href="https:&#x2F;&#x2F;pirsch.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;pirsch.io&#x2F;</a>
评论 #26144906 未加载
评论 #26146500 未加载
评论 #26146034 未加载
评论 #26144002 未加载
评论 #26146062 未加载
评论 #26212480 未加载
tylerjl大约 4 年前
I administer k8s daily at my full-time job and administer a 14-node nomad cluster in my homelab. This accurately captures my sentiments as well. My nomad cluster is even spread across three different CPU architectures (arm, arm64, and amd64) but still works great.<p>One of the points I&#x27;d highlight in this post is just how good the combination of nomad, consul, and consul-template is. Even when nomad lacks some sort of first-class integration that k8s might have, the combination of being able to dynamically generate configuration files using consul-template plus nomad automatically populating the consul catalog means that you can do almost anything, and without much hassle. I use consul-template in the homelab to dynamically wrangle nginx, iptables, and dnsmasq, and it continues to work well years after I initially set it up.<p>I often wish I had the luxury of relying on vault+consul+nomad in all the environments I operate in.
评论 #26146383 未加载
3np大约 4 年前
We moved from k8s to Nomad at my workplace, and I&#x27;m currently running almost all my self-hosted software on a 10-node Nomad cluster (with Consul and Vault) as well. The servers for each of the three gives plenty of headroom resource-wise when run on any recentish arm64 SBC, so you can get an HA cluster for not expensive.<p>If you integrate properly with them (which does take quite a bit of work with the ACL policies and certs), it really starts shining. With terraform, of course.<p>For these core services themselves and other OS packages, I use ansible, mostly because of the huge resources in the community.<p>It&#x27;s fun and doesn&#x27;t come with all the cognitive overhead of k8s. I&#x27;m a fan and will tell everyone they should consider Nomad.<p>It&#x27;s obviously less mature, though. One thing that has been frustrating for a while is the networking configuration - a simple thing like controling which network interface a service should bind to (internal or external?) was supposedly introduced in 0.12 but completely broken until 1.0.2 (current version is 1.0.3).<p>Consul Connect is really awesome conceptually to make a service mesh, but is also just coming together.<p>There are really only two things I miss dearly now:<p>1) exposing consul-connect services made by Nomad (aka ingress gateway). It seems to be theoretically doable but requiring manual configuration of Connect and Envoy. If you want to expose a service from the mesh through e.g. an http load balancer, you need to either expose it raw (losing the security benefits) or manually plumb it (no load balancer seems to play nicely with connect without a lot of work, yet)<p>2) Recognize that UDP is a protocol people actually use in 2021. This is a critique of the whole industry.
评论 #26293495 未加载
评论 #26143898 未加载
mrweasel大约 4 年前
Maybe it&#x27;s because Nomad is a commercial product as well, but it&#x27;s really sad that it doesn&#x27;t get more usage. Kubernetes is the new hotness, to the point where people don&#x27;t care that the don&#x27;t really need Kubernetes. They just know that the big boys are using it, so regardless of scale, so should they.<p>Kubernetes is exceedingly complex, and it needs to be, to support all it&#x27;s use cases. However, I would claim that the vast majority of small project running on Kubernetes is just after one thing: Easier deployments.<p>These users would be just as well served, if not better by Nomad. It eliminates much of the complexity of running a Kubernetes cluster, but it still utilizes hardware better and you get a convenient way to deploy your code.
评论 #26144165 未加载
评论 #26212990 未加载
trabant00大约 4 年前
Nomad looks useful to me for use cases that have nothing to do with &quot;a simple k8s&quot; but more with distributed&#x2F;HA systemd and cron.<p>Just deployed a small 3 node cluster in prod last week for this: run some binaries with some parameters, make sure they restart on failure, have them move to another node if one fails or is rebooted, and don&#x27;t waste resources by having a classical active&#x2F;passive 2 node setup that doesn&#x27;t scale either.<p>It took me a couple of days to read the documentation which is good but not always up to date (I did have to dig in some github issues in one case), create a test setup and check failure scenarios. Gotta say I&#x27;m mostly impressed.
评论 #26143400 未加载
parhamn大约 4 年前
Hashicorp has very cool stuff but I am not a fan of the config language they use on all their projects. It’s fine when I’m in an infrastructure type role and recency recall is fine but when Hashicorps tools are in my periphery it is a pain. Anyone share this? I guess the alternative is templated tomls&#x2F;yamls&#x2F;pseudo-jsons. Wish we’d all agree to one templated configuration format.
评论 #26143503 未加载
评论 #26143329 未加载
评论 #26143354 未加载
评论 #26146057 未加载
评论 #26148291 未加载
评论 #26143157 未加载
评论 #26143627 未加载
zomglings大约 4 年前
From the article: &gt; I always maintain &quot;Day 0 is easy, Day N is the real test of your skills&quot;.<p>Would be interesting to see how this applies to the author&#x27;s use of Nomad. It&#x27;s easy to shit on Kubernetes because of its complexity, but this article seems to be comparing the Nomad Day 0 experience with the Kubenrnetes Day N experience.<p>I&#x27;m firmly of the opinion that you don&#x27;t need much more than systemd (or equivalent) + SSH for your home server.
评论 #26144290 未加载
glynnforrest大约 4 年前
I only have praise for the HashiStack (Nomad, Consul, Vault). Folks often reach for Kubernetes as part of a larger initiative to containerize their apps, add ephemeral test environments, etc, but I really think Nomad is better suited to this.<p>Choosing k8s is just the first step - you have to deal with what &#x27;distribution&#x27; of k8s to use, upgrades, pages of yaml, secrets stored in plaintext by default...<p>Once you&#x27;ve got Nomad running, it just <i>works</i>.<p>I help people move to Nomad, my email is in my profile if you want to chat :)
评论 #26146680 未加载
bithavoc大约 4 年前
Nomad is good at running all sort of workloads, not just container workloads. Also there is a plugin[0] to run containers without docker so Nomad is headed in the right direction for scenarios where you only need a scheduler.<p>[0] <a href="https:&#x2F;&#x2F;github.com&#x2F;Roblox&#x2F;nomad-driver-containerd" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;Roblox&#x2F;nomad-driver-containerd</a>
评论 #26143785 未加载
omneity大约 4 年前
We&#x27;re using Nomad to power several clusters at Monitoro[0].<p>We&#x27;re a super small team and I have to say our experience operating Nomad in production has been extremely pleasant - we&#x27;re big fans of Hashicorp&#x27;s engineering and the quality of their official docs.<p>Our biggest gripe is the lack of managed Nomad offerings such as GKE (on google cloud). However once things are set up and running it requires minimal to no maintenance.<p>I also run it locally for all sorts of services with a very similar experience.<p>As another comment mentioned, it&#x27;s more of a better &#x2F; distributed scheduler such as systemd. The ecosystem of tools around it is the cherry on top (terraform, consul, fabio ...)<p>[0]: <a href="https:&#x2F;&#x2F;monitoro.xyz" rel="nofollow">https:&#x2F;&#x2F;monitoro.xyz</a>
评论 #26143957 未加载
aequitas大约 4 年前
Coincidentally my home server is also named Hydra, running Nomad on CoreOS&#x2F;Flatcar. I&#x27;ve had this setup for years now running without issue (unless I managed to mess it up myself by, for example, changing the IP and not following proper procedures to restore the Raft cluster).<p>Recently I deployed K3s on the same node to add some new workloads, but now that I want to move those workloads to Nomad to get rid of the cpu and memory usage of K8s. I&#x27;m running into what is becoming my main problem with Nomad that I never thought I had.<p>With all it&#x27;s complexities, getting something to run on K8s is as simple as adding a Helm chart name to a Terraform config file and running apply. Maybe I need to set a value or volume, but that&#x27;s it. Everything below is pretty much standardised.<p>With Nomad however, it&#x27;s benefit of doing only one thing very well also means that all the other things like ingress and networking need to be figured out yourself. And since there is no standard regarding these everyone invents their own, preventing something like k8s-at-home [0] to emerge. Also K8s is pretty agnostic in container backend where Nomad needs configurations for every driver.<p>I think writing your own Helm charts for everything would suck more than writing the Nomad configs. Though a lot could be automatically generated for both of them. But I&#x27;m missing a community repository of sorts for Nomad.<p>[0] <a href="https:&#x2F;&#x2F;github.com&#x2F;k8s-at-home&#x2F;charts" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;k8s-at-home&#x2F;charts</a>
评论 #26146038 未加载
OJFord大约 4 年前
&gt; - Job: Job is a collection of different groups. Job is where the constraints for type of scheduler, update strategies and ACL is placed.<p>&gt; - Group: Group is a collection of different tasks. A group is always executed on the same Nomad client node. You&#x27;ll want to use Groups for use-cases like a logging sidecar, reverse proxies etc.<p>&gt; - Task: Atomic unit of work. A task in Nomad can be running a container&#x2F;binary&#x2F;Java VM etc, defining the mount points, env variables, ports to be exposed etc.<p>&gt; If you&#x27;re coming from K8s you can think of Task as a Pod and Group as a Replicaset. There&#x27;s no equivalent to Job in K8s.<p>Is that right? I haven&#x27;t used Nomad (yet?) but, as described, it sounds to me more like Job ~= Deployment; Group ~= Pod; Task ~= Container?
评论 #26145491 未加载
评论 #26146020 未加载
outime大约 4 年前
The biggest problem with these posts is that authors seem to fail to see how K8s isn&#x27;t targeting home servers or small companies. Sure, there&#x27;s K3s which allows you to have a more <i>minimalistic</i> experience but it&#x27;s still K8s.<p>K8s is enterprise stuff and should be seen as such. It&#x27;s complex because the problems that are being attempted to solve are complex and varied. That doesn&#x27;t discount that Nomad is a great piece of software though.
评论 #26142601 未加载
评论 #26146076 未加载
评论 #26142935 未加载
评论 #26143778 未加载
cbushko大约 4 年前
&gt; (Not joking) You are tired of running Helm charts or writing large YAML manifests. The config syntax for Nomad jobs is human friendly and easy to grasp.<p>I write all of my kubernetes resources in terraform because I don&#x27;t want to fight with helm charts. I was going to have to write something to monitor my deployments anyway and alert my co-workers that their deploys failed so why not just use terraform that tells you:<p>- what will change on deploy - fails when a deploy fails - times out<p>I didn&#x27;t want to tell developers to kubectl apply and then watch their pods to make sure everything deployed ok when terraform does this out of the box..
评论 #26146224 未加载
gravypod大约 4 年前
&gt; Increase developer productivity by making it easier to deploy&#x2F;onboard new services.<p>&gt; Consistent experience of deployment by testing the deployments locally.<p>&gt; (Not joking) You are tired of running Helm charts or writing large YAML manifests. The config syntax for Nomad jobs is human friendly and easy to grasp.<p>Seems like most of the ux issues stem from configuring kube easily. I wonder what the author would say about using a metaconfig language like cue or jsonnet to make is super easy to define new workloads.
mads大约 4 年前
We used Nomad on Windows with the raw exec driver to schedule services on a project I worked on a year back. It worked great. Two binaries on each host along with a configuration file (for Nomad and Consul) and you are up and running.<p>Since then I have been a fan. Also worked a lot with Kubernetes, which has its merits, but the simplicity of Nomad is great.
berkayozturk大约 4 年前
In my previous job, I used Nomad and Consul to deploy and scale my machine learning models on Linux and Windows VMs without using any container technology. It was so fun and easy to work with.
bbss大约 4 年前
&gt; host_network as of present cannot bind to a floating IP address (DigitalOcean&#x2F;GCP etc). I&#x27;ve to resort to using my droplet&#x27;s public IPv4 address for now.<p>You should be able to bind to the anchor ip to make the floating ip work. <a href="https:&#x2F;&#x2F;www.digitalocean.com&#x2F;docs&#x2F;networking&#x2F;floating-ips&#x2F;how-to&#x2F;find-anchor-ips&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.digitalocean.com&#x2F;docs&#x2F;networking&#x2F;floating-ips&#x2F;ho...</a>
评论 #26150391 未加载
nikisweeting大约 4 年前
I really liked Nomad last time I tried it (~2 years ago) but I was driven away by the subtle inconsistencies between Nomand, Consul, and Vault&#x27;s HCL configs. Things that should be the same in all three (e.g. bind port, simple permissions options, etc. standard boilerplate) inexplicably had different config keys or HCL structures between them.<p>Maybe this has improved these days, I&#x27;ll have to give it another shot.
awinter-py大约 4 年前
this is compelling:<p>&gt; Nomad shines because it follows the UNIX philosophy of &quot;Make each program do one thing well&quot;. To put simply, Nomad is just a workload orchestrator. It only is concerned about things like Bin Packing, scheduling decisions.<p>so is the stuff about saner config syntax<p>&#x27;k8s is bad at everything and nomad is only trying to be bad at one thing&#x27; actually makes sense of my reality
dr_faustus大约 4 年前
I looked at Nomad about a year ago but deployment of a secure production cluster involved more moving parts than I liked. Especially the security seemed to be very complicated (with a private CA, certificates, Vault, etc. necessary). A simple shared secret would have sufficed imo, but that was not available as an option.
turtlebits大约 4 年前
I ran a Nomad cluster a few years ago and the documentation wasn&#x27;t great for solving problems. I ran into some issues with several cluster nodes, and wasn&#x27;t able to properly troubleshoot.<p>I guess this an ecosystem problem, but in general I find Hashcorp documentation (SRE&#x2F;Ops&#x2F;Admin specific) lacking.
rektide大约 4 年前
A whole lot of anti- Kubernetes &quot;you most likely don&#x27;t need them and the increase in complexity makes it a pain to maintain&quot; and &quot;Kubernetes is exceedingly complex&quot; in this thread &amp; somewhat in this article.<p>I agree that you probably don&#x27;t need Kubernetes, and perhaps yeah it could be considered complex.<p>But I think it&#x27;s the right fit for most developers&#x2F;doers &amp; over time most operators too. Kubernetes is not Kubernetes. Kubernetes is some base machinery, yes, but it&#x27;s also a pattern, for writing controllers&#x2F;operators that take Kubernetes Objects and turn them into things. Take a Postgres object and let postgres-operator turn it into a running, healing, backing-up replicated postgres cluster. Take a SQS object and let ACK turn it into a real SQS. Take a PersistentVolume and with Rook turn it into a Ceph store.<p>Kubernetes &amp; cloud native in general proposes that you should have working models for the state of your world. In addition to the out-of-the-box machinery you get for running containers (deployment-sets), exposing them (services), &amp;c, you get this pattern. You get other folks building operators&#x2F;controllers that implement this pattern[1]. You get a consistent, powerful, extensible way of building.<p>Nothing else comes close. There&#x27;s nothing remotely as interesting in the field right now. The Cult of Easy is loud &amp; bitterly angry about Kubernetes, hates it&#x27;s &quot;complexity&quot;, but what is actually complex is having a dozen different operational environments for different tools &amp; systems. What is actually complex is operating systems yourself, rather than having operators to maintain systems. Kubernetes has some initial costs, it can feel daunting, but it is radically simpler in the long run because _it has a paradigm,_ an all inclusive paradigm that all systems can fit into, and the autonomic behaviors this paradigm supports radically transfer operational complexity from human to computer, across that broad&#x2F;all-inclusive range of systems.<p>There&#x27;s a lot of easier this&#x2F;harder that. No one tries to pitch Nomad or anything else as better, as deeper, as being more consistent, having a stronger core. Every article you hear on an alternative to Kubernetes is 98% &quot;this was easier&quot;. I think those people, largely, miss the long game, the long view. A system that can adapt, that operationally can serve bigger &amp; bigger scopes, ought to pay dividends to you as years go by. Kubernetes may take you longer to get going. But it is time enormously well spent, that will increase your capability &amp; mastery of the world, &amp; bring you together with others building radically great systems whether at home[2][3] or afar. It will be not just a way of running infrastructure, but help you re-think how you develop, and how to expose your own infrastructure &amp; ideas more consistently, more clearly, in the new pattern language of autonomic machines that we have only just begun to build together.<p>I encourage the bold explorers out there, learn Kubernetes, run Kubernetes. And to those of you pitching other things, please, I want you to talk up your big game better, tell me late-game scenarios, tell me how your system &amp; I are going to grow together, advance each other.<p>[1] <a href="https:&#x2F;&#x2F;kubernetes.io&#x2F;docs&#x2F;concepts&#x2F;architecture&#x2F;controller&#x2F;#controller-pattern" rel="nofollow">https:&#x2F;&#x2F;kubernetes.io&#x2F;docs&#x2F;concepts&#x2F;architecture&#x2F;controller&#x2F;...</a><p>[2] <a href="https:&#x2F;&#x2F;github.com&#x2F;onedr0p&#x2F;home-cluster" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;onedr0p&#x2F;home-cluster</a><p>[3] <a href="https:&#x2F;&#x2F;github.com&#x2F;k8s-at-home&#x2F;awesome-home-kubernetes" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;k8s-at-home&#x2F;awesome-home-kubernetes</a>
评论 #26145769 未加载
评论 #26145898 未加载
评论 #26146898 未加载
superkuh大约 4 年前
Why would you run Nomad for a home server? This isn&#x27;t for home server use. This is for practicing skills only required in commercial environments. Skills you only have a need for because someone pays you: because it&#x27;s not fun and stupid.<p>Taking that process home and doing all that instead of just running nginx from your distro repos on the sbc OS is cargo cult insanity for 99% of cases.
评论 #26149833 未加载
评论 #26144480 未加载
评论 #26144153 未加载