We decided to move 90% of our workload from the cloud to on-prem infrastructure

&gt; <i>&quot;Starting a web-based or SaaS (Software as a Service) business was virtually unheard of before the age of IaaS&quot;</i><p>Nonsense. There were plenty of SaaS startups. There was even a little event called the dotcom boom all about internet companies. This lack of history and experience is why new companies get into this cloud-first mess in the first place.<p>Cloud is primarily for flexibility in iteration, dynamic scaling, or complex configurations that would be otherwise hard to do. If you have steady-state load like this then a few servers in your colo is pretty simple and far cheaper.<p>Companies also vastly overestimate their scale when their entire business could probably fit on a single commodity server.

I work in HPC for a cloud provider, and fully endorse this move. Anonymously, of course.<p>You can make an economic argument for or against cloud in practically every IT domain, but in HPC the case for on-prem is really compelling; none of the cloud networking&#x2F;resiliency value-add is relevant to batch workflows, and costs per core-hour are only remotely comparable if you use spot - which is itself a major compromise.<p>The only real advantage cloud has for science is object storage, which is genuinely a much better idea than trying to manage your own long-term archival storage.<p>If I were independent I would recommend people buy and build on-prem clusters and shuffle data out of fast scratch into Glacier, but other than that just don&#x27;t worry about cloud until price pressure kicks in and we are down to 1-2 cents per core-hour on-demand.<p>I&#x27;d love a role where I can say these things non-anonymously, but the salary for such a position would be at least 50% lower than working for a cloud provider. Keep that in mind when talking to your supplier - we may not believe the pitch ourselves, but making it is just part of the job.

Wholeheartedly agree. After my cloud storage costs exploded (mostly S3 egress over to Hetzner&#x2F;OVH), I noticed that renting a 1GBit&#x2F;s fiber connection to my office is actually quite affordable at $80 per month (in northern Germany).<p>Of course, it&#x27;s not globally distributed and there is no fail-safe, but for the work we&#x27;re doing, that is no issue. If we employees are offline, it doesn&#x27;t matter if our tools are offline, too. And now that all AI storage is local anyway, building a GPU compute node is easy. I&#x27;m still waiting for 3090 prices to drop further, though, in contrast to the article. But I also went with Ryzon 5950 and Linux. I was positively surprised that 10G fiber networking is now down to $70 for a PCIe card + 20m cable kit. My workstation now has 1010MB&#x2F;s 4k random write on the network filesystem. (We used SAMBA 3.1 and CIFS mounts)<p>I also grabbed the python&#x2F;ubuntu package lists off Google Colab and created my own Docker to imitate it, and now data processing and AI training is fast (I always get the good GPU, no luck involved) and dirt cheap. Originally the idea was to run it on OVH, but I&#x27;m now also running it locally.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;fxtentacle&#x2F;ovh-colab-sagemaker-compatibility-mode&#x2F;blob&#x2F;master&#x2F;Dockerfile" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;fxtentacle&#x2F;ovh-colab-sagemaker-compatibil...</a>

Just want to highlight how futuristic the author&#x27;s title is: &quot;Computational Biologist, Head of Protein Design @ Enzymit&quot;.<p>Why they moved to on-prem: lower and more predictable cost. At a public cloud provider, they lost thousands of dollars (of free credit they had) through &quot;architectural blunders&quot;. And the running cost of GPU, CPU, storage, and data transfer summed up to $10K a month - at which point they figured they might as well purchase their own compute servers.

The flexibility and scalability of the cloud comes at a cost. Scientific, static and non web but IO heavy workloads are almost always better off run on-prem or in a co-located data centre with servers paid up front. As is most things in businesses, it’s a trade-off one needs to think about and make. The smart ones will come out ahead, and those that blindly follow FAANG and drink their kool-aid may or may not come ahead, albeit with a heavy hit to their(or their VC’s) pocket.<p>If you’re a startup that simply have a bunch of web apps and APIs where uptime and network are your major costs, on-prem is only going to become a worthless headache.<p>A good Systems Engineer should help to figure out such choices. Anybody need one?

I hate the fact that hiring now basically requires cloud experience specific to vendors. This is basically going to force people into one of the three major cloud platforms.

&gt; We do not have (yet) any public-facing applications that need to scale across multiple geographical zones and handle millions of requests per minute.<p>Most don&#x27;t. 1mm requests per minute is very pedestrian for a single vm in virtually all cases. 1mm per <i>second</i> is totally reasonable too if you are careful with a few things...<p>I genuinely believe you could put the literal public Netflix biz experience on a single VM. Account management, billing, preferences, view history, etc. The only pieces that need cloud scale are ddos mitigation, 4k video streams and media-dense static web content. Most businesses do not have strong need for these things.

&lt;tinfoil hat&gt; I always got the impression that there has been a lot of cloud propaganda&#x2F;astroturfing, even on HN.<p>I&#x27;m seeing more of these &quot;on-prem infrastructure&quot; posts, citing costs, efficiency, and cloud complexity. We run most of our infra on-prem, and have looked to moving a few bits and pieces to the cloud, but the math almost always works out to buying hardware. Meanwhile, I talk to some &lt;cloud stack&gt; friends, and their opex costs are astronomical for the traffic and size of their products.<p>The cloud is extremely convenient, and I would choose it if I were launching a new product, but past certain sizes and expenses I would start to do some math. It&#x27;s not terribly difficult to run these cloud &quot;shrinkwrapped&quot; products (such as load balancers) on-prem. Things like object storage seem more difficult to me. I&#x27;m also hesitant to admin a database, they intimidate me :)

Hope he&#x27;s prepared for a letter from nVidia&#x27;s lawyers for breaking their license agreement for the RTX3090&#x27;s.

On premise can be a lot cheaper for bandwidth intensive app. We have a 10gbit&#x2F;s dedicated fiber we use at like 80%, it cost us 1500$ per month (power+fiber). This kind of bandwidth is 10 to 100 times more expensive on the cloud depending on the service. Of course, no CDN, but as our customers are mostly local, we don&#x27;t need global presence.<p>Also, we serve this from a single epyc based server, using elixir&#x2F;phoenix. It&#x27;s at about 6gbit&#x2F;s outbound traffic. I realize this is not uber redundant, but it works and keep the costs low.

We came to the same conclusion.<p>We do hosing for commercial HVAC systems and due to software requirements and the human factor of training the VM&#x27;s we had in azure cost us significantly more than on premises servers. Add to that we generally keep those servers a little longer than some others would just increases the savings. This is still true even though we pay a colocation to host those servers.<p>The cloud flexibility is completely irrelevant for us in light of multi year service contracts from each customer we work with.

Why did openstack fail? Or did it, was it just not adopted?<p>I think there is still a lot of potential for open source management of core EC2&#x2F;S3&#x2F;networking capabilities (aka &quot;core AWS IAAS service&quot;). We have a fair number of cloud abstraction layers now, and obviously kubernetes, you&#x27;d think we could as an industry produce core apis for doing resource listing, availability, etc.<p>Maybe some of the problem is that devs have a LOT of experience with the &quot;ask&quot; side of IaaS: gimme storage, gimme vms, set. But they have no experience with the &quot;provide&quot; side, and the sort of one-off manual nature of installing networking and machines doesn&#x27;t have good standardization for &quot;reporting available resources&quot;.<p>At this point, aws apis are somewhat stable. (I would bitch about the error codes and documentation... but anyway). It&#x27;s obviously &quot;good enough&quot; after 10-15 years of them.<p>Are there projects that try to marry an aws-ish api, which really is a reporting and request api, with a &quot;available resources&quot; reporting api? Are some of these things out there?<p>AWS ten years ago was liberating. It was progress. It was a good thing. But Amazon is not a &quot;do no evil&quot; corporation, much the opposite. And you see this in AWS with its treatment of startups, open source projects, and other manipulations. They are a monopoly now, or at a minimum a dangerous cartel.<p>A real open source alternative would be a good thing. It would be good for the rest of FAANG, it would encourage competition by allowing lesser clouds to offer core competencies that are drop-in.

<i>Starting a web-based or SaaS (Software as a Service) business was virtually unheard of before the age of IaaS (Infrastructure as a Service) companies. There were simply too many hurdles, and the expenses were too high — you’d have to purchase dedicated servers and a high bandwidth connection to handle a load of incoming visitors to your website, hire engineers to build a scalable system, and if you planned to go international, you would have to purchase servers in other geographical locations.</i><p>The first internet boom was exactly what the author describes. I worked for an internet e-commerce start-up in 1999, and guess what? We had dedicated hosting bandwidth coming into our office, and the hardware and software in place to serve our application. Of course, one of the founders was a sysadmin, but I had friends working for similar start-ups, and they leveraged one of the many co-location providers in the city to manage all the details. Yet their employer still owned actual hardware that they could touch when necessary.

<p><pre><code> After spending $100k in a year.. After doing the math, We decided to purchase three workstations at a total cost of $17k. One is a GPU-based workstation with two RTX 3090s and an Intel i9–12900 CPU, and another two workstations with 16 cores AMD Ryzen 5950X CPUs. It took us a few FTE days to set those up to our satisfaction with slurm, NFS, backups, and several other services. We noticed that our RTXs, although considered gaming cards, are comparable (if not better) in performance to Tesla V100, which some cloud providers rent at the staggering price of $3.06 an hour. </code></pre> tl;dr - They did the math.

As a random aside, I’m glad to see ‘on-prem’ emerge as a common shorthand for this, only because it always grates me to see people make the extremely common mistake of saying ‘on-premise’. <i>Premise</i>, of course, only ever means “an idea or theory on which a statement or action is based”, whereas the actual term is <i>premises</i> (“the land and buildings owned by someone, especially by a company or organisation”, as in “The security guards escorted the protesters off the premises”).

This is a good explanation of cloud issues for a company with resources and consistent workload...<p>IaaS is not really competitive in this space, I don&#x27;t think... If you have access to system admins, and have a consistent work load, you could avoid the cloud, trading the cloud premium for more employees and skills in your team. This is fine, if that is what you&#x27;re team needs.<p>The cloud is not a silver bullet that solves all companies infrastructure... But they have a very profitable space, especially in small businesses, or businesses that benefit from multiple data centers. AWS simply made it easy to scale up and down, as well as scale around the world... if you don&#x27;t need to dynamically scale or easy access to multiple data centers, the cloud begins to lose it&#x27;s best (cost-effective) competitive edge to self hosting... Though at the micro scale, the cloud can do dead simple basics for free -- or near enough (e.g. static websites), which is fun for personal projects

&gt;Adding storage, CPU time, and many other costs makes understanding and verifying the cost structure a task suitable for certified experts<p>Is that easier on-prem? I was under the impression it was even more difficult--especially with shared tenancy (how much incremental cost does App A auth add to our Active Directory deployment?)<p>You&#x27;re going to need to know server power utilization under load to calculate power&#x2F;cooling costs and probably some additional data on network utilization to figure out incremental costs for that<p>Maybe if it&#x27;s a colo or managed data center that gets rolled up for you, but if you&#x27;re managing yourself, you still have to figure it out<p>Blog post also doesn&#x27;t mention cost of downtime (maybe not an issue for them) or a metrics solution (you usually get basic machine and service metrics for free on the big cloud providers)

1) You save a metric boatload of money this way<p>2) Onprem experience looks bad on your resume, compared to cloud experience<p>3) Incentives work for people, and resume-driven development is key, as our industry is very stingy in passing the savings from 1) to the developers

&quot;First, it’s important to note that Enzymit’s use of cloud computing mainly entailed computationally intensive calculations for protein design&quot;<p>So they got a very specific workload that is easy to run on any infra including on-prem.

5 years ago we evaluated cold storage for 100 TB of data. Even at that relatively small amount magnetic tapes were much more cheaper even after accounting for an extra copy to ship to a remote location.

Yeah, for on-prem scientific work, heavy CPU usage, probably bursty, no need for all the fancy things, security etc. - this is a pretty straight forward case for on-prem use.<p>Or not even on-prem, just renting some physical boxes from some place where they&#x27;re only going to have a basic markup.<p>You have to wait a week to get few more boxes but that&#x27;s not a big deal.<p>This is the kind of thing I&#x27;d imagine a corp would start doing pretty early, because the &#x27;flexibility&#x27; of IaaS just isn&#x27;t worth the cost.

There are a lot of hidden costs for not using a cloud too. One of them is security. Frankly, you won&#x27;t come up with anything better than AWS IAM in the short term. If you don&#x27;t care much about that, then it is a different story, but I am happy to pay the premium knowing that all data is adequately encrypted and all lambda functions have the minimum permissions required to run.

If you are paying the cloud premium make sure you actually utilize the advantages it offers – global distribution, elasticity, rapid scaling, high availability, disaster recovery. Otherwise what&#x27;s the point? If your needs can be met by shoving a couple thousand dollars worth of hardware into a closet somewhere that is obviously going to be a lot cheaper.

&gt;&gt; &quot;Starting a web-based or SaaS (Software as a Service) business was virtually unheard of before the age of IaaS&quot;<p>&gt; &quot;...off-load many of its processes to an <i>on-premise private cloud</i>?&quot;<p><i>On-Premise Private Cloud</i><p>I dont know why reading this my brain just cant compute. You mean you have three consumer grade computer with Dedicated GPU running 24x7?

It is a question of the firm&#x27;s model. A one time cost + electricity - depreciation or a recurring, variable monthly cost that may exceed the the cost of the hardware? I have two used workstations that cost less than 6 months of GCE time. Does that work for me? yes might not work for others. (especially if the workstation dies)

The calculation is very thin in my opinion.<p>10k of infra costs are not a lot of money in a business context.<p>A person to operate a private cloud with OnCall, backup hardware, capacity planing etc. costs what?<p>I would always try to have GPU on prem as those prices are quite high bit others I would use managed.<p>Cloud providers are just much better in operating infrastructure ad normal ops.

Many companies that do this look at MinIO for object storage. Given they run in AWS, GCP and Azure, they will minimize or eliminate your application rewrites. They are cloud-native by design and very fast.

&gt; After doing the math, We decided to purchase three workstations at a total cost of $17k.<p>Why not share the math? Isn&#x27;t this article supposed to answer why it&#x27;s more economical to use on-prem?

It is on-premises not on-premise. Why is this so hard?

The issue I see is that the public clouds often offer carbon free or carbon offset power. When you deploy onprem... This often gets ignored.