Escaping surprise bills and over-engineered messes: Why I left AWS

Over the years I’ve spent a lot of time talking engineers and managers out of using serverless AWS options for various reasons. I’ve found that most non-infra focused engineers and managers see serverless marketed as “simpler” and “cheaper”.<p>It’s often the opposite, but most people don’t see that until after they’ve built their infrastructure around that, get locked in, and then start seeing the surprise bills, difficult to diagnose system failures, and hard-limitations start rolling in.<p>A bit of early skepticism, and alternative solutions with a long-term perspective in mind, often go a long way.

I see these articles on HN enough almost always end up agreeing. My gross over simplification of this problem is:<p><pre><code> The sweet spot for AWS seemingly is US$300-$3k&#x2F;mth. </code></pre> At that low end, you can get a mix of basic-but-okay DB, web servers, caching, CDN and maybe a few Lambdas all with logging&#x2F;x-ray. All you&#x27;ll need to run a good site&#x2F;service w&#x2F; low-medium traffic and solid reliability. At around $3k&#x2F;mth, you likely know whether you&#x27;re in AWS to stay (using many services) or are eyeing the self hosting route.<p>Side projects really need to optimize under this $5-50&#x2F;mth and that&#x27;s just too low on AWS. The expensive foot-guns are just too risky.

Truth is, most web projects made today can run on a raspberry pi or mini PC and be just fine. If you have enough users that you need to scale to more machines, you&#x27;ll be in a position to know what to do to handle it, or hire someone who does.

&gt; One app gave me trouble - a Python Flask app. It had several complicated dependencies including OpenCV, Numpy, and Matplotlib. My belief is that this had more to do with the complex nature of the libraries and less to do with NFS.<p>It has to do with the allround lunacy that surrounds the python ecosystem the moment you step outside the stdlib nursery. I picked up python because I wanted to contribute to a project. Was immediately met with ** of the first order.<p>After decades of C-like languages, I like (some of) the syntax. But I hate the ecosystem and having to ride a bullock cart after cruising in a plane. The language is slow. So all the critical work is done in C&#x2F;Rust. And then you write python bindings for your library. Once that is available, you can &quot;program in python.&quot;<p>The dependencies are a nightmare, particularly if you do anything LLM-related. Some libraries simply won&#x27;t work with newer versions of the interpreter. After decades of Java, this came as a big surprise.<p>If it were not for uv, I might have given up on the language altogether.

Personally I use lightsail on AWS and cloudflare cause there is always an off ramp to try some of the fancy stuff but then you can always go back to just using cheap VMs behind cloudflare. You can also put it all behind a VPC and you can use CDK&#x2F;CloudFormation so that’s also nice.<p>I gave up on using GCP even though the products like BigQuery are way better just because I got burned too many times like with the Google Domains -&gt; Squarespace transition.<p>I’m thinking of switching back to a bare metal provider now like Vultr or DO (would love to know what people are using these days I haven’t used bare metal providers since ~2012).<p>Also, completely unrelated does anyone know what the best scraping proxy is these days for side projects (data journalism, archiving etc.)?

I&#x27;ve used NearlyFreeSpeech for years (as registrar &amp; DNS), and I&#x27;ve loved their service. Their site is plain, and you just trade money for a plain, simple product, with basically 0 bullshit between you and that exchange. Their site is so refreshing in today&#x27;s landscape of upsells and other corporate dark patterns.<p>The article implicates AWS, but AFAICT the other major cloud, GCP, behaves similarly. The docs for &quot;budget alerts&quot;[1] do call it out directly,<p>&gt; <i>Setting a budget does not automatically cap Google Cloud or Google Maps Platform usage or spending. Budgets trigger alerts to inform you of how your usage costs are trending over time. Budget alert emails might prompt you to take action to control your costs, but they don&#x27;t automatically prevent the use or billing of your services when the budget amount or threshold rules are met or exceeded.</i><p>But still. But wait, you say, those docs go on to suggest,<p>&gt; <i>One option to automatically control spending is to use budget notifications to programmatically disable Cloud Billing on a project.</i><p>And the linked page states,<p>&gt; <i>Following the steps in this capping example doesn&#x27;t guarantee that you won&#x27;t spend more than your budget.</i><p><i>sigh</i> &quot;Over-Engineered Messes&quot;, TFA hits it on the nose.<p>There&#x27;s also limiting API usage, but that&#x27;s on requests … not on cost.<p>I avoid it all for personal stuff.<p>At work, we pipe all these cloud bills into a BigQuery account which then pipes into graphs in Grafana, which all tells us that engineers have no idea what the actual horsepower of 3 GHz * 32 cores is when they request a bajillion more cores.<p>It&#x27;s probably also reasonably categorized as an &quot;Over-Engineered Mess&quot;.<p>(We also import Azure&#x27;s billing data, and boy do they make that obnoxious. CSV files dumping into a bucket, and they might go back &amp; edit CSVs, or drop new ones, and <i>if</i> there is there a schema for those CSV files … I&#x27;ve yet to find it. Some columns you might think were <i>guaranteed</i> non-&quot;&quot; are not. Dates in American. Severely denormalized. Etc.)<p>[1]: <a href="https:&#x2F;&#x2F;cloud.google.com&#x2F;billing&#x2F;docs&#x2F;how-to&#x2F;budgets" rel="nofollow">https:&#x2F;&#x2F;cloud.google.com&#x2F;billing&#x2F;docs&#x2F;how-to&#x2F;budgets</a>

I am self hosting on a NAS at home, with Cloudflare in front (which does the most of the work) and Cloudflare tunnels to avoid exposing anything directly. The tunnel communicates with various Docker instances depending on the services.<p>It works flawlessly for now, and costs almost zero, since the NAS is always on in any case, and Cloudflare is free.<p>These are all small projects of course, but two of them stayed on HN frontpage for a day and didn&#x27;t break a sweat.

Talk about burying the lede: &quot;My bill has increased from about $1 to $7 a month.&quot;<p>I agree with much of the sentiment, but I don&#x27;t see how complex things you could possibly be making things if you&#x27;re paying $1 a month ...

That is pretty cool. I tend to default to AWS, luckily not too expensive for my side projects (about $15&#x2F;month) - nothing accessible to the public though so my cost is relatively predictable.<p>That said, I do wish you could hard shutdown at a certain budget limit.. but guess that is not in AWS’s best interested.

I was using Azure for a cloud windows desktop which I need rarely, for using something like itunes or kobo to download books. It cost ~$5 a month.<p>One day windows update bricks the system (hanging while trying to revert some security patch), and over a couple months from time to time I try random janky official azure recovery tools and voodoo from user forums with people who don&#x27;t really know what they&#x27;re doing either.<p>I notice my bill has crept up to several hundred dollars a month. Each of the recovery tools was cloning the system + disks, and I ended up with a bunch of disks that chewed up my bill.<p>I raised a support ticket and they refunded part of it with a bit of &quot;you&#x27;re a bad person&quot;, but wow... although the primary lesson I got here is that I never want to use windows again.

AWS definitely has its place, but for personal projects, the complexity and cost risk can be overkill. The horror stories of surprise bills are real—misconfigured services, forgotten instances, and unexpected data transfer fees can add up fast. Even with alerts, by the time you notice, it’s often too late.<p>For those who need AWS but want to avoid these surprises, there are ways to keep costs in check. Cost allocation tags, savings plans, and budget alerts help, but they require ongoing effort. Tools like SpendShrink.com can automate cost analysis and highlight savings opportunities before they become an issue.<p>It’s great to see more people looking for simpler hosting solutions, but for those who do need AWS, better cost visibility is a must.

You can get 1Gbps unlimited traffic VPS on IONOS 12 vCores, 24GB RAM, 640GB storage for $50&#x2F;month.<p>No need to pay 9 cents per GB egress to the big clouds.<p><a href="https:&#x2F;&#x2F;www.ionos.com&#x2F;servers&#x2F;cloud-vps" rel="nofollow">https:&#x2F;&#x2F;www.ionos.com&#x2F;servers&#x2F;cloud-vps</a>

Other good option for hardcaps is to put as much as possible on bunny cdn<p>Still means the dynamic logic need to go elsewhere but for basic storage it works well for prepay

Found the article light on details here.<p>What kind of AWS setup leads to surprise bills? If your blog ends up on front page of HN how much is the traffic spike going to cost? $100? $1000? $10000?<p>Ofc I&#x27;ve seen the horror stories of recursive lambdas causing 1M+ in compute costs but for a small site I&#x27;m unsure what kind of issues we&#x27;re talking about here...<p>If pages are mostly static doesn&#x27;t a CDN make most of this go away?

Most developers aren&#x27;t sysadmins and most sysadmins don&#x27;t work closely with their developers.<p>After you factor that in most of the disasters of this type involving the cloud or home spun rust are easily explained away.<p>The former are missing skills that should have been beet into them by any decent education and the later is normally overwhelmed and in need of a team to replace them.

AWS isn’t and has never been economical for side projects or hobby tinkering, unless you <i>specifically</i> want to tinker with AWS.<p>I’m a big AWS fan. I’d recommend any company of decent size to use AWS. But seriously, if your project is just a personal blog or some rails app you tinker with on the weekend, just get a $5&#x2F;mo instance on Digital Ocean or a raspberry pi.

I’m very confused by this blog post. The risk of exploding costs he’s talking about comes from an attack, right?<p>So he doesn’t have the risk appetite for a DDOS attack nor the desire to figure out how to properly mitigate one. And this is AWS’s fault?

I use Lambda + DynamoDB to run a very basic app for managing access to my servers. The monthly cost is $0.02 - $0.05.<p>I&#x27;m also looking into using S3 glacier for my server backups. It appears to be the cheapest option.

I like that the service he recommends, NearlyFreeSpeech, will host anything that is legal. But they do require your real identity info. Is there a way to anonymously host websites today?

I am lucky I have access to a company AWS account, otherwise I would never have started to learn their ecosystem. I cannot afford the personal risk of a massive unexpected bill.

I think it&#x27;s fairly obvious why people try to escape aws. What is far less clear is why folks try to embrace their own solutions when this clearly ain&#x27;t their forte

I don&#x27;t get it, you can set as many budget alerts as you like on AWS. If you get an email you can decide to stop the service.

I don&#x27;t know why it&#x27;s so hard to convince tech leadership other hosting options are possible.

I don&#x27;t think &quot;surprise bills&quot; is a good reason to avoid AWS. Yes, you avoid them on other providers by doing simple bare-metal Linux boxes. But doing a bare-metal Linux box with no other services is also fixed-cost at AWS. You only get into tricky to determine in advance variable costs when you string together a bunch of AWS&#x27;s extra services in ways that you don&#x27;t understand well or don&#x27;t&#x2F;can&#x27;t set limits on.<p>On the other hand, doing single bare-metal boxes being much more complex and usually more expensive is a good reason to skip AWS for simple projects. In addition to the profusion of instance types and billing&#x2F;usage options.<p>I also think the lack of options on limiting max billing for flexible services is pretty reasonable actually. For most of them, there&#x27;s no single obvious reasonable thing to do when the money hits the limit. Storage costs money too, but I don&#x27;t think much of anyone really wants to have their data get deleted when the cost hits the limit, for example.

Reads like an ad.

Ironically, NearlyFreeSpeech still didn&#x27;t exactly provide the &quot;simple monthly max spend option&quot; you were looking for, because now you&#x27;re maintaining multiple prepaid accounts. It&#x27;s silly that online services won&#x27;t provide a billable account with a hard max.