The cost of cloud, a trillion dollar paradox

Am I alone in not seeing a paradox here? It&#x27;s no different from a variety of things that make sense at some scales and not at others.<p>Seems similar to office space - a 5 person company will find the economics of coworking space compelling, a 500 person company will do better with a long-term lease with a commercial landlord, and a 50,000 person company might have their own property management team in-house. That doesn&#x27;t create a paradox in the commercial real estate market, just different solutions for different needs.

Linden Lab, the people behind Second Life, may have made this mistake. They had leased data center space in Phoenix AZ, and owned their own machines, about 5000 of them. During 2020, the whole system was moved to AWS.<p>Most of the servers, each of which handles an area of the virtual world, run all the time, regardless of user load. This is very different than web serving. It&#x27;s the worst cost case for AWS, where the real benefits come from dynamically provisioning a changing load. AWS, used for dedicated servers, can cost 2x that of owning your own. One data center operator says 4x.[1]<p>Before the conversion, I&#x27;d asked the responsible VP if they were sure AWS wasn&#x27;t going to be more expensive. He retired when the conversion was complete. Now we hear rumors of the company finding out that AWS costs are higher than the old data center.<p>[1] <a href="http:&#x2F;&#x2F;www.happi.io&#x2F;aws-instances-versus-physical-servers&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.happi.io&#x2F;aws-instances-versus-physical-servers&#x2F;</a>

The opportunity cost is staggering. Not just of the people that do the work, but the cost of focus and top-level priorities over the years it takes to do the work.<p>Everybody loves to cite DropBox here. The greater arc of DropBox is their product stalled, they lost the enterprise of their market to Box, and they found themselves a commodity in a commodity market. Heck, maybe they&#x27;ll go back to Amazon if they keep doing things like this: <a href="https:&#x2F;&#x2F;aws.amazon.com&#x2F;solutions&#x2F;case-studies&#x2F;dropbox-s3&#x2F;" rel="nofollow">https:&#x2F;&#x2F;aws.amazon.com&#x2F;solutions&#x2F;case-studies&#x2F;dropbox-s3&#x2F;</a><p>But that&#x27;s not all. It takes a top-level directive to repatriate an entire SaaS. That&#x27;s at the cost of other top-level projects. It&#x27;s wild to me that any company that has significant fuel left in the tank would buy back single-digit COGS percentages instead of investing in product that could add double-digit growth for several years at scale.

This has been widely known for many years. The crossover point happens much sooner than I think people intuit but most companies never really measure or model it. My experience at a few different companies is that DIY infrastructure pencils out at 30-40% of the cost of the cloud.<p>There is a learned helplessness when it comes to companies running their own data centers that has become widespread over the last decade. What used to be a fairly mechanical process has almost been mythologized as some kind of arcane art beyond the technical ability of any company that isn&#x27;t Google or Amazon. Designing data center builds isn&#x27;t difficult, it is a pretty straightforward albeit detail-oriented blue-collar engineering skill, but it seems few people learn it anymore.

If it needs to be said, this is more or less exactly the thesis behind Oxide[0], and matches what we are seeing in the market. It&#x27;s certainly validating to see a VC firm echo our pitch deck back to us, even if one that (in)famously doesn&#x27;t believe in hardware! ;)<p>[0] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27294471" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27294471</a>

Not trying to peddle cheap skepticism here.<p>This article only looks at &quot;seen&quot; costs, and assumes that there are no &quot;unseen&quot; costs to running on-prem. Many companies do not have the operational maturity to run on-prem well. The result: high cost of operations, low availability, and large increase in time-to-value.<p>Second unseen cost: everybody becomes their own SI. So far nobody really sells the &quot;whole stack&quot; for running on prem. I mean hardware, network, virtualization, application, traffic mgmt, etc. I have to buy stuff from two dozen different vendors and cobble it together into a high-labor, rickety Jenga tower of stuff.

This analysis completely skips over the &quot;elastic&quot; aspect of cloud infrastructure, which was a key motivator for using these providers since the very beginning and is as relevant today regardless of company size.<p>My company (which is in fact part of the charts in the article) had every single metric across the board spike 15-20x basically overnight when the pandemic started last year in March. Our entire infrastructure burden was clicking a few buttons on the AWS console and making sure everything was provisioning and scaling as needed. If we had to send out people to buy hard drives and server racks at that time, there is no chance we would have been able to meet the extra demand.<p>Plus, if you give me a few dozen capable engineers today I&#x27;m not going to waste their efforts on rebuilding AWS to get a best-case few percentage point return on our cloud spend. I&#x27;ll launch a new product for our customers instead.

&quot;But that’s just Dropbox.&quot;<p>The implication here being, &quot;Well, if Dropbox can save this much, then think about how much everyone else cans save!&quot; But in fact the opposite is true. Dropbox sells disk storage on the cloud. For them to do so by effectively <i>reselling</i> someone else&#x27;s disk-storage-on-the-cloud platform is obviously not high margin, and they&#x27;d be better off building it themselves. So, sure. Anyone else also offering, disk storage on the cloud, or compute clusters on the cloud, or otherwise just reselling someone else&#x27;s product on the cloud, will probably have higher margins doing it themselves. But that is certainly not most companies. So, yeah, it&#x27;s &quot;just Dropbox&quot;.<p>Disclaimer: I work for a cloud provider, but these opinions are my own.

Disclaimer: I&#x27;m a Co-Founder at Vantage, a cloud cost platform. I also worked in public cloud for ~6 years at AWS, DigitalOcean, etc.<p>While repatriation can make sense at a larger scale company, startups and SMBs can yield the same benefits discussed in this post by simply tracking and optimizing cloud spend.<p>We try to make this as easy as possible for people with <a href="https:&#x2F;&#x2F;www.vantage.sh&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.vantage.sh&#x2F;</a> - where we&#x27;re already helping thousands of individuals, startups, SMBs and enterprises as it relates to AWS.

In 2012, I was working on a migration from EC2 to on-prem &amp; was working on infrastructure building for zCloud. The cost factor was huge, the switch out of EC2 literally paid for itself in a lot of ways.<p>The work was very interesting, because a lot of it was actually building a private cloud for internal customers &amp; the work primarily centered around a virtualized data-center aimed at boom-bust cycles of games (15+ million users for 6 weeks, drop to 2 million for a month and down to a million in another week).<p>The issue is that the infrastructure cost is somewhat constant when dealing with that sort of fluctuations in revenues, so the cost to revenue ratio was unpredictable (while the cost was).<p>So what happened in the end looked a lot like a fire-sale of hardware when the cost was unbearable, while if it was an end-user cloud, that low-demand phase would be able to cut losses as a spot instance or something.<p>Anyway, a few years after I left, back to EC2 it is[1].<p>[1] - <a href="https:&#x2F;&#x2F;aws.amazon.com&#x2F;solutions&#x2F;case-studies&#x2F;zynga&#x2F;" rel="nofollow">https:&#x2F;&#x2F;aws.amazon.com&#x2F;solutions&#x2F;case-studies&#x2F;zynga&#x2F;</a>

These people don&#x27;t see the real value proposition of the cloud. The value is not in &quot;scaling when you need to&quot;. Sure, that can be very handy. But that is not what 99% of companies are getting out of the cloud 99% of the time.<p>If you self-manage, your capital investment is initially higher, and lower over time. At the same time, the effort it takes to reach the same results is <i>always</i> higher, and the quality of the end product <i>may</i> be lower, depending on how much service quality affects your product.<p>If you pay for managed services, your initial investment is lower, and higher over time. But at the same time, you require less effort, and you get higher quality outcomes.<p>This is obvious to anyone who has worked in the industry and done both. First, host your own service: JFrog Artifactory, Atlassian Confluence, GitLab, whatever. Now rapidly increase the demands on this service. As demand rises, quality will decrease, because it takes a lot of time, effort and expertise to build a very reliable hosted service. Now switch to a managed cloud instance. Suddenly, the service&#x27;s average quality increases. Performance is steady regardless of increase in use. There are virtually no interruptions to your product or development.<p>The impact of a service&#x27;s quality and reliability has ripple effects. If poor service quality slows down development, that means development quality will go down as people cut corners to try and meet deadlines. If the service is used for production, it means your product&#x27;s quality will suffer, and that effects your bottom line. So a huge amount of the actual cost is not just paying for a service, but also how much business value is generated or lost due to service quality.<p>There is simply no way to replicate a managed service without becoming a managed service provider yourself. You have to become a whole new business within a business. It&#x27;s like a yogurt company also becoming a dairy farm. Running a farm is not easy, and you will screw it up for several years. Seems obvious for farming, but for some reason people always underestimate this when it comes to technology.<p>On paper, the Cloud&#x27;s value proposition is scalability. But in practice, the true value is actually as a force-multiplier for your product&#x27;s quality, reliability, and time to market. (Time to market not just being &quot;I launched my startup&quot; but also &quot;I released this new feature before my competitor&quot;)

One benefit of &quot;the cloud&quot; not mentioned here is <i>elapsed time to acquire a new instance</i><p>If you have moved to AWS &#x2F; other then that time is around 5 minutes.<p>If you are in a major fortune 500 and need a new server, quite often that time will measure in months (yes really).<p>This simple equation just blows every other cost&#x2F;benefit calculation out of the water.<p>I may have missed it in the discussion.

As I read it i couldnt help but think the following<p>Yes, but the benefit of cloud is we only work to optimize those which gain market adoption. For every twilio there may be 100-1000 startups that did not make it, it&#x27;s a good thing if those were constructed rapidly, tested for product market fit and then turned off without optimizations applied.<p>Also if cloud adoption actually accelerates building, then it may also aid its adopters if there is a race to product market fit.

This is not surprising. It&#x27;s kind of obvious that cloud costs will scale more dramatically than any other cost center. So it&#x27;s really an attractive idea to think that you can cut your largest bill by going on-prem.<p>The problem with going on-prem is that for most companies this is so complex and hard to operationalize that they will likely just fail at that attempt.<p>Cloud biggest contribution besides seamless infrastructure access, is the way it has lower the overall technical competency threshold to operate an internet business. It&#x27;s really easier to find someone that knows AWS or Azure than is to find that someone who knows how to build on-prem infrastructure.<p>I imagine that repatriation for the average large cloud business will only be possible for those who are already operating in modern infrastructure constructs like Kubernetes.<p>If you have critical workflows build on propietary cloud services, you&#x27;re probably fucked. You will have to rebuild every single service from scratch and make it scalable out of the box. If you&#x27;re a multi-dimensional business that sounds like a nightmare.<p>I honestly think that Dropbox was capable of doing it just because the nature of their business.<p>At the primitive level you&#x27;re mostly replacing your storage vendor. It happens to be that storage is their business model so it&#x27;s quite evident that repatriating that critical piece of your business will yield great gains. I don&#x27;t think it&#x27;s that simple for other cloud businesses.

It isnt really a paradox at all. Its specifically the opportunity that companies like Oxide will go after. Basically the stage of your business determines the best course of action. Youd be insane to start with large datacenters and high capex for an undefined ROI for most projects. Similarly you dont understand your requirements or workloads yet, which would also affect the efficiency of your architecture.<p>Whenever a rewrite happens in software there are usually massive performance gains. The same thing happens with infrastructure.

&gt; [...] paradox: You’re crazy if you don’t start in the cloud; you’re crazy if you stay on it.<p>&gt; So what can companies do to free themselves from this paradox? As mentioned, we’re not making a case for repatriation one way or the other; rather, we’re pointing out that infrastructure spend should be a first-class metric. What do we mean by this? That companies need to optimize early, often, and, sometimes, also outside the cloud. When you’re building a company at scale, there’s little room for religious dogma.

Many underestimate the complexity and the headache that comes with managing its own data center. It is easy to get a couple of racks up and running. Then you have to deal:<p>- HW that fails, go get a new one, cost + time<p>- backup and geographical redundancy<p>- security<p>- certifications<p>- tooling to manage and forecast<p>- black friday! scale for few days ... more HW?<p>- etc ...<p>It is way more convenient for the average enterprise to pay premium and be done with it. So they can focus on the actual business and not building infrastructure and tooling to manage it.

What should be considered RIGHT NOW, is that given the supply chain issues, if you are NOT on a cloud platform, good luck getting new hardware from major vendors. With AWS&#x2F;Azure you can provision immediately. If you have some old hardware whose performance is impacting your production environment given the dependency on it, good luck. Firms like Dell are already stating this.

It&#x27;s not purely a technical matter, there are different market dynamics at play as well.<p>If you go cloud, you get a handful of very large suppliers, that provide a lot of non-standardized services that are probably going to lock you in like hell (as the article wel says).<p>If you build infra, you&#x27;re using &#x2F;mostly&#x2F; commoditized supplies and skills.<p>If the cloud offerings where interchangeable and the industry reasonably fragmented, the margins of cloud providers would be slimmer and the paradox would probably go away (in favor of: always cloud!).<p>See for example Porter&#x27;s analysis framework [1] and how your positioning changes in the two cases.<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Porter%27s_five_forces_analysis" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Porter%27s_five_forces_analysi...</a>

This may be a naive question, but...<p>Given that interactions to the cloud are through a relatively small set of known APIs, I&#x27;m surprised that there isn&#x27;t there a service which replicates the APIs but with the endpoints being on &quot;repatriated&quot; hardware.

&gt; Because this shift happens later in a company’s life, it is difficult to reverse as it’s a result of years of development focused on new features, and not infrastructure optimization.<p>Little Miss Muffet<p>Sits on her tuffet<p>Watching her billing grow<p>Her devs ever hectoring<p>On the need for refactoring<p>But what do these uni-brows know?

A bit of a weird article coming from a VC heavily invested in SaaS providers

Cloud is not just virtualized compute, storage and network (csn) anymore. Most of the services in AWS&#x2F;GCP offer a lot more than just virtualized csn. I dont think, the article factors in the cost of running a service. Its one thing to rent csn and run your own database. Its completely another to use a cloud service like dynamodb&#x2F;bigtable etc. I agree, opex cost can go haywire, unless kept in check. But, its not an apples to apple comparison.

I think they understated the importance and difficulty of retaining a sufficiently redundant skilled workforce to manage the equivalent cloud infrastructure.

I think one of the benefits of cloud is (in theory) the agility that it brings. So much is abstracted away that you truly can start to focus on what&#x27;s more important to your business. There&#x27;s a cost to that value. So for example, while the cost to serve may be lower with your own infrastructure, it may be that the opportunity costs go up and your ability to grow that revenue stream or customer base is diminished without cloud.<p>Another thing that came to mind reading this was - what would I need to make such a switch (from cloud to your own data centre)? And then I thought, you could provide a business doing this - commoditised hardware and services, providing the real-estate and the engineers, and more importantly, providing the same kind of service stack you find in the cloud. Then I thought, that could be a cool business for someone. And then I thought - there&#x27;s nothing stopping Amazon or Microsoft or Google from offering that.<p>I guess if you have a very clear understanding of your needs, a mature appreciation for what it will take, and a good handle on your likely growth, then taking control of it all may make sense. But for a lot of business (especially those not actually tech businesses per se), it would more likely be a return to pain.

This isn&#x27;t surprising at all. This industry is incredibly fad driven. Cloud became the fad, and the buzz was that cloud saves money, so everyone goes cloud, and then cloud eventually costs more than the original stuff did.<p>The same is happening with SaaS. SaaS eliminates the need for in-house IT! Except it doesn&#x27;t. It just means you now have a bunch of recurring SaaS costs that you are locked into forever because they have your data and you still need IT people to babysit your massive cloud&#x2F;SaaS sprawl.<p>Not following fads and buzz is a <i>huge</i> competitive advantage in this industry. Founders and chief engineers &#x2F; CTOs &#x2F; CIOs take note. Just make sure you can explain why you are not using (insert latest buzzword here).<p>The bottom line is that you should analyze the situation using <i>your</i> work load, <i>your</i> numbers, <i>your</i> culture, etc., and decide what works the best. Sometimes that&#x27;s managed cloud. Sometimes it&#x27;s unmanaged cloud. Sometimes it&#x27;s bare metal. Sometimes it&#x27;s on-prem. Your mileage will vary.

From a Moore&#x27;s Law perspective I&#x27;d like to see a true cost of ownership over time, as most infra goes obsolete in 18-24 months.<p>Public clouds, like AWS, have cut their storage costs by more than 50% since Dropbox built their own infra in 2015.

I see services like Storj the perfect hybrid model here. Companies get the security and optimizations of the cloud, while being able to simultaneously run their own storage nodes to get cost savings.<p>Exciting times for decentralized storage!

If cloud margins are really 50%, that seems like a case for new competition in the cloud business rather than making a data center for your own non cloud business.

Is this really a paradox?<p>When you&#x27;re small, outsource.<p>Should the company find traction, and as you gain size, the arguments in favor of outsourcing fade, and bringing the IT in-house makes sense.

this isn&#x27;t a paradox to me, this has always been the case.... big companies can optimize more. They have more non-optimal points due to sheet size alone

I don’t think is a paradox, is a standard “make or buy” decision which is different based on your company’s size and growth phase.

&gt; By tracking cloud spend, the company enables engineers, and not just finance teams, to take ownership of cloud spend.<p>&gt; tie the pain directly to the folks who can fix the problem<p>That&#x27;s why we&#x27;re building <a href="https:&#x2F;&#x2F;github.com&#x2F;infracost&#x2F;infracost" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;infracost&#x2F;infracost</a> for engineering teams (free open source)

I couldn’t tell, which onprem portfolio company are they pushing here?

technologies like kubernetes make repatriation even simple in some cases.

&quot;We show (using relatively conservative assumptions!) that across 50 of the top public software companies currently utilizing cloud infrastructure, an estimated $100B of market value is being lost among them due to cloud impact on margins — relative to running the infrastructure themselves.&quot;<p>That completely overlooks the actual benefit and the reasons driving cloud adoption, which he states early on and then fails to integrate here. The real cost&#x2F;benefit analysis would take into account the amount of time, money, and opportunity cost saved by using a cloud in development, a critical time that determines whether there&#x27;ll actually be a profitable company eventually. Optimizing the cloud costs is certainly important, but being able to spin up highly integrated systems on demand offsets a huge amount of time and capital during development.<p>A takeaway might be that, e.g., AWS, should offer even larger discounts for large scale operations in order to retain mature customers.