TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Why we use our own hardware

933 pointsby nmjenkins5 months ago

47 comments

jph005 months ago
The original answer to &quot;why does FastMail use their own hardware&quot; is that when I started the company in 1999 there weren&#x27;t many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70&#x2F;month. There weren&#x27;t really practical VPS or SaaS alternatives back then for what I needed.<p>Rob (the author of the linked article) joined a few months later, and when we got too big for our Rackspace server, we looked at the cost of buying something and doing colo instead. The biggest challenge was trying to convince a vendor to let me use my Australian credit card but ship the server to a US address (we decided to use NYI for colo, based in NY). It turned out that IBM were able to do that, so they got our business. Both IBM and NYI were great for handling remote hands and hardware issues, which obviously we couldn&#x27;t do from Australia.<p>A little bit later Bron joined us, and he automated absolutely everything, so that we were able to just have NYI plug in a new machine and it would set itself up from scratch. This all just used regular Linux capabilities and simple open source tools, plus of course a whole lot of Perl.<p>As the fortunes of AWS et al rose and rose and rose, I kept looking at their pricing at features and kept wondering what I was missing. They seemed orders of magnitude more expensive for something that was more complex to manage and would have locked us into a specific vendor&#x27;s tooling. But everyone seemed to be flocking to them.<p>To this day I still use bare metal servers for pretty much everything, and still love having the ability to use simple universally-applicable tools like plain Linux, Bash, Perl, Python, and SSH, to handle everything cheaply and reliably.<p>I&#x27;ve been doing some planning over the last couple of years on teaching a course on how to do all this, although I was worried that folks are too locked in to SaaS stuff -- but perhaps things are changing and there might be interest in that after all?...
评论 #42488986 未加载
评论 #42488720 未加载
评论 #42489018 未加载
评论 #42488700 未加载
评论 #42488669 未加载
评论 #42489783 未加载
评论 #42491824 未加载
评论 #42492967 未加载
评论 #42541547 未加载
评论 #42488600 未加载
评论 #42488635 未加载
评论 #42492150 未加载
评论 #42489015 未加载
评论 #42492196 未加载
评论 #42488783 未加载
评论 #42492279 未加载
评论 #42492400 未加载
评论 #42519944 未加载
johnklos5 months ago
The whole push to the cloud has always fascinated me. I get it - most people aren&#x27;t interested in babysitting their own hardware. On the other hand, a business of just about any size that has any reasonable amount of hosting is better off with their own systems when it comes purely to cost.<p>All the pro-cloud talking points are just that - talking points that don&#x27;t persuade anyone with any real technical understanding, but serve to introduce doubt to non-technical people and to trick people who don&#x27;t examine what they&#x27;re told.<p>What&#x27;s particularly fascinating to me, though, is how some people are so pro-cloud that they&#x27;d argue with a writeup like this with silly cloud talking points. They don&#x27;t seem to care much about data or facts, just that they love cloud and want everyone else to be in cloud, too. This happens much more often on sites like Reddit (r&#x2F;sysadmin, even), but I wouldn&#x27;t be surprised to see a little of it here.<p>It makes me wonder: how do people get so sold on a thing that they&#x27;ll go online and fight about it, even when they lack facts or often even basic understanding?<p>I can clearly state why I advocate for avoiding cloud: cost, privacy, security, a desire to not centralize the Internet. The reason people advocate for cloud for others? It puzzles me. &quot;You&#x27;ll save money,&quot; &quot;you can&#x27;t secure your own machines,&quot; &quot;it&#x27;s simpler&quot; all have worlds of assumptions that those people can&#x27;t possibly know are correct.<p>So when I read something like this from Fastmail which was written without taking an emotional stance, I respect it. If I didn&#x27;t already self-host email, I&#x27;d consider using Fastmail.<p>There used to be so much push for cloud everything that an article like this would get fanatical responses. I hope that it&#x27;s a sign of progress that that fanaticism is waning and people aren&#x27;t afraid to openly discuss how cloud isn&#x27;t right for many things.
评论 #42487146 未加载
评论 #42487859 未加载
评论 #42487023 未加载
评论 #42488358 未加载
评论 #42487449 未加载
评论 #42486671 未加载
评论 #42488779 未加载
评论 #42487132 未加载
评论 #42486596 未加载
评论 #42486836 未加载
评论 #42488442 未加载
评论 #42486859 未加载
评论 #42487736 未加载
评论 #42487872 未加载
评论 #42487401 未加载
评论 #42487493 未加载
评论 #42487635 未加载
评论 #42488068 未加载
评论 #42486929 未加载
评论 #42486528 未加载
评论 #42486629 未加载
评论 #42486875 未加载
评论 #42487245 未加载
评论 #42486693 未加载
评论 #42487982 未加载
评论 #42486619 未加载
评论 #42488126 未加载
评论 #42487444 未加载
评论 #42486772 未加载
评论 #42486590 未加载
评论 #42487454 未加载
评论 #42487650 未加载
评论 #42487272 未加载
评论 #42488180 未加载
评论 #42486726 未加载
评论 #42486741 未加载
评论 #42487785 未加载
评论 #42487353 未加载
评论 #42487657 未加载
评论 #42487605 未加载
评论 #42487538 未加载
评论 #42495591 未加载
评论 #42487059 未加载
评论 #42487244 未加载
评论 #42488079 未加载
评论 #42492441 未加载
评论 #42488278 未加载
lukevp5 months ago
To me, Cloud is all about the shift left of DevOps. It’s not a cost play. I’m a Dev Lead &#x2F; Manager and have worked in both types of environments over the last 10 years. It’s immeasurable the velocity difference as far as system provisioning between the two approaches. In the hardware space, it took months to years to provision new machines or upgrade OSes. In the cloud, it’s a new terraform script and a CI deploy away. Need more storage? It’s just there, available all the time. Need to add a new firewall between machines or redo the network topology? Free. Need a warm standby in 4 different regions that costs almost nothing but can scale to full production capacity within a couple of minutes? Done. Those types of things are difficult to do with physical hardware. And if you have an engineering culture where the operational work and the development work are at odds (think the old style of Dev &#x2F; QA &#x2F; Networking &#x2F; Servers &#x2F; Security all being separate teams), processes and handoffs eat your lunch and it becomes crippling to your ability to innovate. Cloud and DevOps are to me about reducing the differentiation between these roles so that a single engineer can do any part of the stack, which cuts out the communication overhead and the handoff time and the processes significantly.<p>If you have predictable workloads, a competent engineering culture that fights against process culture, and are willing to spend the money to have good hardware and the people to man it 24x7x365 then I don’t think cloud makes sense at all. Seems like that’s what y’all have and you should keep up with it.
评论 #42489309 未加载
评论 #42488873 未加载
评论 #42489041 未加载
评论 #42489891 未加载
评论 #42489539 未加载
_bare_metal5 months ago
Plugging <a href="https:&#x2F;&#x2F;BareMetalSavings.com" rel="nofollow">https:&#x2F;&#x2F;BareMetalSavings.com</a><p>in case you want to ballpark-estimate your move off of the cloud<p>Bonus points: I&#x27;m a Fastmail customer, so it tangentially tracks<p>----<p>Quick note about the article: ZFS encryption can be flaky, be sure you know what you&#x27;re doing before deploying for your infrastructure.<p>Relevant Reddit discussion: <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;zfs&#x2F;comments&#x2F;1f59zp6&#x2F;is_zfs_encryption_bug_still_a_thing&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;zfs&#x2F;comments&#x2F;1f59zp6&#x2F;is_zfs_encrypt...</a><p>A spreadsheet of related issues that I can&#x27;t remember who made:<p><a href="https:&#x2F;&#x2F;docs.google.com&#x2F;spreadsheets&#x2F;d&#x2F;1OfRSXibZ2nIE9DGK6swwBZXgXwdCPKgp4SbPZwTexCg&#x2F;htmlview" rel="nofollow">https:&#x2F;&#x2F;docs.google.com&#x2F;spreadsheets&#x2F;d&#x2F;1OfRSXibZ2nIE9DGK6sww...</a>
评论 #42489886 未加载
评论 #42486579 未加载
评论 #42490009 未加载
bartvk5 months ago
Such an awesome article. I like how they didn&#x27;t just go with the Cloud wave but kept sysadmin&#x27;ing, like ol&#x27; Unix graybeards. Two interesting things they wrote about their SSDs:<p>1) &quot;At this rate, we’ll replace these [SSD] drives due to increased drive sizes, or entirely new physical drive formats (such E3.S which appears to finally be gaining traction) long before they get close to their rated write capacity.&quot;<p>and<p>2) &quot;We’ve also anecdotally found SSDs just to be much more reliable compared to HDDs (..) easily less than one tenth the failure rate we used to have with HDDs.&quot;
评论 #42485888 未加载
评论 #42489894 未加载
评论 #42488033 未加载
评论 #42485916 未加载
评论 #42486825 未加载
akpa15 months ago
The fact that Fastmail work like this, are transparent about what they&#x27;re up to and how they&#x27;re storing my email and the fact that they&#x27;re making logical decisions and have been doing so for quite a long time is exactly the reason I practically trip over myself to pay them for my email. Big fan of Fastmail.
评论 #42493519 未加载
评论 #42487561 未加载
DarkCrusader25 months ago
I have seen a common sentiment that self hosting is almost always better than cloud. What these discussions does not mention is how to effectively run your business applications on this infrastructure.<p>Things like identity management (AAD&#x2F;IAM), provisioning and running VMs, deployments. Network side of things like VNet, DNS, securely opening ports etc. Monitoring setup across the stack. There is so much functionalities that will be required to safely expose an application externally that I can&#x27;t even coherently list them out here. Are people just using Saas for everything (which I think will defeat the purpose of on-prem infra) or a competent Sys admin can handle all this to give a cloud like experience for end developers?<p>Can someone share their experience or share any write ups on this topic?<p>For more context, I worked at a very large hedge fund briefly which had a small DC worth of VERY beefy machines but absolutely no platform on top of it. Hosting application was done by copying the binaries on a particular well known machine and running npm commands and restarting nginx. Log a ticket with sys admin to create a DNS entry to point a reserve and point a internal DNS to this machine (no load balancer). Deployment was a shell script which rcp new binaries and restarts nginx. No monitoring or observability stack. There was a script which will log you into a random machine for you to run your workloads (be ready to get angry IMs from more senior quants running their workload in that random machine if your development build takes up enough resources to effect their work). I can go on and on but I think you get the idea.
评论 #42489917 未加载
评论 #42487133 未加载
xiande045 months ago
Aside: Fastmail was the best email provider I ever used. The interface was intuitive and responsive, both on mobile and web. They have extensive documentation for everything. I was able to set up a custom domain and and a catch-all email address in a few minutes. Customer support is great, too. I emailed them about an issue and they responded within the hour (turns out it was my fault). I feel like it&#x27;s a really mature product&#x2F;company and they really know what they&#x27;re doing, and have a plan for where they&#x27;re going.<p>I ended up switching to Protonmail, because of privacy (Fastmail is within the Five Eyes (Australia)), which is the only thing I really like about Protonmail. But I&#x27;m considering switching back to Fastmail, because I liked it so much.
评论 #42487837 未加载
评论 #42487767 未加载
TheFlyingFish5 months ago
Lots of people here mentioning reasons to both use and avoid the cloud. I&#x27;ll just chip in one more on the pro-cloud side: reliability at low scale.<p>To expand: At $dayjob we use AWS, and we have no plans to switch because we&#x27;re <i>tiny</i>, like ~5000 DAU last I checked. Our AWS bill is &lt;$600&#x2F;mo. To get anything remotely resembling the reliability that AWS gives us we would need to spend tens of thousands up-front buying hardware, then something approximating our current AWS bill for colocation services. Or we could host fully on-prem, but then we&#x27;re paying even more up-front for site-level stuff like backup generators and network multihoming.<p>Meanwhile, RDS (for example) has given us something like one unexplained 15-minute outage in the last six years.<p>Obviously every situation is unique, and what works for one won&#x27;t work for another. We have no expectation of ever having to suddenly 10x our scale, for instance, because we our growth is limited by other factors. But at our scale, given our business realities, I&#x27;m convinced that the cloud is the best option.
评论 #42488365 未加载
评论 #42488371 未加载
nisa5 months ago
Love this article and I&#x27;m also running some stuff on old enterprise servers in some racks somehwere. Now over the last year I&#x27;ve had to dive into Azure Cloud as we have customers using this (b2b company) and I finally understood why everyone is doing cloud despite the price:<p>Global permissions, seamless organization and IaC. If you are Fastmail or a small startup - go buy some used dell poweredge with epycs in some Colo rack with 10Gbe transit and save tons of money.<p>If you are a company with tons of customers, ton&#x27;s of requirements it&#x27;s powerful to put each concern into a landing zone, run some bicep&#x2F;terraform - have a ressource group to control costs and get savings on overall core-count and be done with it.<p>Assign permissions into a namespace for your employe or customer - have some back and forth about requirements and it&#x27;s done. No need to sysadmin across servers. No need to check for broken disks.<p>I&#x27;m also blaming the hell of vmware and virtual machines for everything that is a PITA to maintain as a sysadmin but is loved because it&#x27;s common knowledge. I would only do k8s on bare-metal today and skip the whole virtualization thing completly. I guess it&#x27;s also these pains that are softened in the cloud.
mgaunard5 months ago
Why is it surprising? It&#x27;s well known cloud is 3 times the price.
评论 #42486130 未加载
评论 #42486170 未加载
jmakov5 months ago
Would be interesting to know how files get stored. They don&#x27;t mention any distributed FS solutions like SeaweedFS so once a drive is full, does the file get sent to another one via some service? Also ZFS seems an odd choice since deletions (esp of small files) at +80% full drive are crazy slow.
评论 #42486279 未加载
评论 #42485891 未加载
评论 #42486267 未加载
评论 #42489929 未加载
louwrentius5 months ago
I like this writeup, informative and to-the-point.<p>Today, the cloud isn’t about other people’s hardware.<p>It’s about infrastructure being an API call away. Not just virtual machines but also databases, load-balancers, storage, and so on.<p>The cost isn’t the DC or the hardware, but the hours spend on operations.<p>And you can abuse developers to do operations on the side :-)
评论 #42485876 未加载
goldeneye13_5 months ago
Didn’t see this in the article, do they have multi az redundancy? I.e. if the entire raid goes up in flames what’s the recovery process?
评论 #42486240 未加载
评论 #42489073 未加载
评论 #42489029 未加载
评论 #42486098 未加载
Beijinger5 months ago
I was told Fastmail is excellent, and I am not a big fan of gmail. Once locked out for good in gmail, your email and apps associated with it, are gone forever. Source? Personal experience.<p>&quot;A private inbox $60 for 12 months&quot;. I assume it is USD, not AU$ (AFAIK, Fastmail is based in Australia.) Still pricey.<p>At <a href="https:&#x2F;&#x2F;www.infomaniak.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.infomaniak.com&#x2F;</a> I can buy email service for an (in my case external) domain for 18 Euro a year and I get 5 inboxes. And it is based in Switzerland, so no EU or US jurisdiction.<p>I have a few websites and fastmail would just be prohibitive expensive for me.
评论 #42487745 未加载
评论 #42488007 未加载
评论 #42487849 未加载
评论 #42488191 未加载
indulona5 months ago
I am working on a personal project(some would call it startup, but i have no intention of getting external financing and other americanisms) where i have set up my own cdn and video encoding, among other things. These days, whenever you have a problem, everyone answers &quot;just use cloud&quot; and that results in people really knowing nothing any more. It is saddening. But on the other hand it ensures all my decades of knowledge will be very well paid in the future, if i&#x27;d need to get a job.
tiffanyh5 months ago
FYI - Fastmail web client has Offline support in beta right now.<p><a href="https:&#x2F;&#x2F;www.fastmail.com&#x2F;blog&#x2F;offline-in-beta&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.fastmail.com&#x2F;blog&#x2F;offline-in-beta&#x2F;</a>
评论 #42488057 未加载
评论 #42493487 未加载
评论 #42486759 未加载
rmbyrro5 months ago
if you don&#x27;t have high bandwidth requirements, like for background &#x2F; batch processing, the ovh eco family [1] of bare metal servers is incredibly cheap<p>[1] <a href="https:&#x2F;&#x2F;eco.ovhcloud.com&#x2F;en&#x2F;" rel="nofollow">https:&#x2F;&#x2F;eco.ovhcloud.com&#x2F;en&#x2F;</a>
caidan5 months ago
I absolutely love Fastmail. I moved off of Gmail years ago with zero regrets. Better UI, better apps, better company, and need I say better service? I still maintain and fetch from a Gmail account so it all just works seamlessly for receiving and sending Gmail, so you don’t have to give anything up either.
评论 #42486303 未加载
评论 #42486196 未加载
评论 #42488166 未加载
评论 #42486138 未加载
评论 #42486116 未加载
ackshi5 months ago
I&#x27;m a little surprised it seems they didn&#x27;t have some existing compression solution before moving to zfs. With so much repetitive text across emails I would think there would be a LOT to gain, such as from dictionaries, compressing many emails into bigger blobs, and fine-tuning compression options.
评论 #42486550 未加载
评论 #42493526 未加载
antihero5 months ago
I’ve started to host my own sites and stuff on an old MacBook in a cupboard with a shit old external hardware Ava microk8s and it’s great!
评论 #42486921 未加载
throw0101b5 months ago
&gt; <i>So after the success of our initial testing, we decided to go all in on ZFS for all our large data storage needs. We’ve now been using ZFS for all our email servers for over 3 years and have been very happy with it. We’ve also moved over all our database, log and backup servers to using ZFS on NVMe SSDs as well with equally good results.</i><p>If you&#x27;re looking at ZFS on NVMe you may want to look at Alan Jude&#x27;s talk on the topic, &quot;Scaling ZFS for the future&quot;, from the 2024 OpenZFS User and Developer Summit:<p>* <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=wA6hL4opG4I" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=wA6hL4opG4I</a><p>* <a href="https:&#x2F;&#x2F;openzfs.org&#x2F;wiki&#x2F;OpenZFS_Developer_Summit_2024" rel="nofollow">https:&#x2F;&#x2F;openzfs.org&#x2F;wiki&#x2F;OpenZFS_Developer_Summit_2024</a><p>There are some bottlenecks that get in the way of getting all the performance that the hardware often is capable of.
ttul5 months ago
I think mailbox hosting is a special use case. The primary cost is storage and bandwidth and you can indeed do better on storage and bandwidth than what Amazon offers. That being said, if Fastmail asked Amazon for special pricing to make the move, they would get it.
neeeeeeal5 months ago
What not many people talk about in the comments is how the hardware route is fairly stacked against smaller players. Large enterprises buy the same hardware as small and midsize businesses at a fraction of the cost, which significantly impacts the economics of this decision. Even if you have the capability and desire, if each server costs your business double what an enterprise would pay, it becomes less attractive pretty quickly.
kayson5 months ago
Any ideas how they manage the ZFS encryption key? I&#x27;ve always wondered what you&#x27;d do in an enterprise production setting. Typing the password in at a prompt as any seem scalable (but maybe they have few enough servers that it&#x27;s manageable) and keeping it in a file on disk or on removable storage would seem to defeat the purpose...
herf5 months ago
zfs encryption is still corrupting datasets when using zfs send&#x2F;receive for backup (huge win for mail datasets), would be cautious about using it in production:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;openzfs&#x2F;zfs&#x2F;issues&#x2F;12014">https:&#x2F;&#x2F;github.com&#x2F;openzfs&#x2F;zfs&#x2F;issues&#x2F;12014</a>
评论 #42493541 未加载
评论 #42488304 未加载
评论 #42489996 未加载
kwakubiney5 months ago
If I remember correctly, StackOverflow does something similar. The then Director of Engineering speaks about it on here[1]<p>[1]<a href="https:&#x2F;&#x2F;hanselminutes.com&#x2F;847&#x2F;engineering-stack-overflow-with-roberta-arcoverde" rel="nofollow">https:&#x2F;&#x2F;hanselminutes.com&#x2F;847&#x2F;engineering-stack-overflow-wit...</a>
评论 #42489845 未加载
Axsuul5 months ago
Anyone know what are some good data centers or providers to host your bare metal servers?
评论 #42488381 未加载
veidr5 months ago
&quot;WHY we use our own hardware...&quot;<p>The why is is the interesting part of this article.
评论 #42487107 未加载
tuananh5 months ago
gmail does spam filtering very well for me. fastmail on the other hands, puts lots of legit emails into spam folder. manually marking &quot;not spam&quot; doesn&#x27;t help<p>other than that, i&#x27;m happy with fastmail.
评论 #42486408 未加载
评论 #42493559 未加载
评论 #42486289 未加载
EdJiang5 months ago
I was a bit confused by the section on backups. How do they manage moving the data offsite with the on-premises backup servers? Wouldn’t that be a cost savings by going cloud?
IYasha5 months ago
Very, very reasonable! And the HDD vs. SSD part is just reading my thoughts. :)
xsc5 months ago
Are those backups geographically distributed?
评论 #42488052 未加载
briHass5 months ago
The biggest win with running your own infra is disk&#x2F;IO speeds, as noted here and in DHH&#x27;s series on leaving cloud (<a href="https:&#x2F;&#x2F;world.hey.com&#x2F;dhh&#x2F;we-have-left-the-cloud-251760fb" rel="nofollow">https:&#x2F;&#x2F;world.hey.com&#x2F;dhh&#x2F;we-have-left-the-cloud-251760fb</a>)<p>The cloud providers really kill you on IO for your VMs. Even if &#x27;remote&#x27; SSDs are available with configurable ($$) IOPs&#x2F;bandwidth limits, the size of your VM usually dictates a pitiful max IO&#x2F;BW limit. In Azure, something like a 4-core 16GB RAM VM will be limited to 150MB&#x2F;s across all attached disks. For most hosting tasks, you&#x27;re going to hit that limit far before you max out &#x27;4 cores&#x27; of a modern CPU or 16GB of RAM.<p>On the other hand, if you buy a server from Dell and run your own hypervisor, you get a massive reserve of IO, especially with modern SSDs. Sure, you have to share it between your VMs, but you own all of the IO of the hardware, not some pathetic slice of it like in the cloud.<p>As is always said in these discussions, unless you&#x27;re able to move your workload to PaaS offerings in the cloud (serverless), you&#x27;re not taking advantage of what large public clouds are good at.
评论 #42487271 未加载
beaugunderson5 months ago
I don&#x27;t trust anything from fastmail after they bought pobox and forced me onto their new service which fails at the one thing pobox did well--forwarding email. They also refused to give me a refund (prorated or not) for removing the product I was using and substituting a defective one.
评论 #42493523 未加载
lakomen5 months ago
You also terminate accounts at your sole discretion
awinter-py5 months ago
everyone is &#x27;cattle not pets&#x27; except the farm vet who is shoulder-deep in a cow<p>(my experience with managed kubernetes)
0xbadcafebee5 months ago
I&#x27;ve been doing this job for almost as long as they have. I work with companies that do on-prem, and I work with companies in the cloud, and both. Here&#x27;s the low down:<p>1. The cost of the server is not the cost of on-prem. There are so many different <i>kinds</i> of costs that aren&#x27;t just monetary. (&quot;we have to do more ourselves, including <i>planning, choosing, buying, installing, etc,</i>&quot;) Those are tasks that require expertise (which 99% of &quot;engineers&quot; do not possess at more than a junior level), and time, and staff, and correct execution. They are much more expensive than you will ever imagine. Doing any of them wrong will causes issues that will eventually cost you business (customers fleeing, avoiding). That&#x27;s much worse than a line-item cost.<p>2. You have to develop relationships for good on-prem. In order to get good service in your rack (assuming you don&#x27;t hire your own cage monkey), in order to get good repair people for your hardware service accounts, in order to ensure when you order a server that it&#x27;ll actually arrive, in order to ensure the DC won&#x27;t fuck up the power or cooling or network, etc. This is not something you can just read reviews on. You have to actually physically and over time develop these relationships, or you will suffer.<p>3. What kind of load you have and how you maintain your gear is what makes a difference between being able to use one server for 10 years, and needing to buy 1 server every year. For some use cases it makes sense, for some it really doesn&#x27;t.<p>4. Look at all the complex details mentioned in this article. These people go <i>deep</i>, building loads of technical expertise at the OS level, hardware level, and DC level. It takes a long time to build that expertise, and you usually cannot just hire for it, because it&#x27;s generally hard to find. This company is very unique (hell, their stack is based on Perl). Your company won&#x27;t be that unique, and you won&#x27;t have their expertise.<p>5. If you hire someone who actually knows the cloud really well, and they build out your cloud env based on published well-architected standards, you gain not only the benefits of rock-solid hardware management, but benefits in security, reliability, software updates, automation, and tons of unique features like added replication, consistency, availability. You get a lot more for your money than just &quot;managed hardware&quot;, things that you literally could never do yourself without 100 million dollars and five years, but you only pay a few bucks for it. The <i>value</i> in the cloud is insane.<p>6. Everyone does cloud costs wrong the first time. If you hire somebody who does have cloud expertise (who hopefully did the well-architected buildout above), they can save you 75% off your bill, by default, with nothing more complex than checking a box and paying some money up front (the same way you would for your on-prem server fleet). Or they can use spot instances, or serverless. If you choose software developers who care about efficiency, they too can help you save money by not needing to over-allocate resources, and right-sizing existing ones. (Remember: you&#x27;d be doing this cost and resource optimization already with on-prem to make sure you don&#x27;t waste those servers you bought, and that you know how many to buy and when)<p>7. The major takeaway at the end of the article is <i>&quot;when you have the experience and the knowledge&quot;</i>. If you don&#x27;t, then attempting on-prem can end calamitously. I have seen it several times. In fact, just one week ago, a business I work for had <i>three days of downtime</i>, due to hardware failing, and not being able to recover it, their backup hardware failing, and there being no way to get new gear in quickly. Another business I worked for literally hired and fired four separate teams to build an on-prem OpenStack cluster, and it was the most unstable, terrible computing platform I&#x27;ve used, that constantly caused service outages for a large-scale distributed system.<p>If you&#x27;re not 100% positive you have the expertise, just don&#x27;t do it.
评论 #42495530 未加载
评论 #42490010 未加载
tucnak5 months ago
Yeah, Cloud is a bit of a scam innit? Oxide is looking more and more attractive every day as the industry corrects itself from overspending on capabilities they would never need.
评论 #42486330 未加载
nprateem5 months ago
Yeah and some people reckon web frameworks are bad too. Sometimes it might make sense to host your on your own hardware but almost certainly not for startups.
评论 #42490014 未加载
oldpersonintx5 months ago
longtime FM user here<p>good on them, understanding infrastructure and cost&#x2F;benefit is essential in any business you hope to run for the long haul
rrgok5 months ago
I would like to know the tech stack behind it.
评论 #42490022 未加载
lokimedes5 months ago
A mail-cloud provider uses its own hardware? Well, that’s to be expected, it would be a refreshing article if it was written by one of their customers.
评论 #42493448 未加载
pammf5 months ago
Cost isn’t always the most important metric. If that was the case, people would always buy the cheapest option of everything.
tndibona5 months ago
But what about the cost and complexity of a room with the racks and the cooling needs of running these machines? And the uninterrupted power setup? The wiring mess behind the racks.
评论 #42486286 未加载
评论 #42486200 未加载
评论 #42486384 未加载
评论 #42486261 未加载
评论 #42486285 未加载
dorongrinstein5 months ago
We at Control Plane (<a href="https:&#x2F;&#x2F;cpln.com" rel="nofollow">https:&#x2F;&#x2F;cpln.com</a>) make it easy to repatriate from the cloud, yet leverage the union of all the services provided by AWS, GCP and Azure. Many of our customers moved from cloud A to cloud B, and often to their own colocation cage, and in one case their own home cluster. Check out <a href="https:&#x2F;&#x2F;repatriate.cloud" rel="nofollow">https:&#x2F;&#x2F;repatriate.cloud</a>
rob_c5 months ago
Hosts online service seems to think deserving of medal for discovering that S3 buckets from a cloud provider are crap and cost a fortune.<p>The heading in this space makes your think they&#x27;re running custom FPGAs such as with Gmail, not just running on metal... As for drive failures, welcome to storage at scale. Build your solution so it&#x27;s a weekly task to replace 10disks at a time not critical at 2am when a single disk dies...<p>Storing&#x2F;Accessing tonnes of &lt;4kB files is difficult, but other providers are doing this on their own metal with CEPH at the PB scale.<p>I love ZFS, it&#x27;s great with per-disk redundancy but CEPH is really the only game in town for inter-rack&#x2F;DC resilience which I would hope my email provider has.
评论 #42490020 未加载