TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Cutting down AWS cost by $150k per year simply by shutting things off

191 点作者 tuananh超过 1 年前

37 条评论

darth_avocado超过 1 年前
It is unfortunate that cost management isn’t something most engineers keep an eye out for on a regular basis. Spinning up unnecessary resources, not cleaning up resources properly once not needed, writing inefficient code, etc. all quickly adds up to hundreds of thousands of dollars per month in big companies.<p>I once found a “test” db cluster from an engineer who hadn’t worked in the company for 3 years. We were paying 300k yearly for it before discounts. It took me a literal click to shut it down. And I’m not proud of it but, had to send out an org wide email on the savings achieved (corporate politics :shrug:).
评论 #39091969 未加载
评论 #39092210 未加载
评论 #39091964 未加载
评论 #39093104 未加载
评论 #39092253 未加载
评论 #39093204 未加载
评论 #39092179 未加载
评论 #39093007 未加载
评论 #39091960 未加载
评论 #39092291 未加载
评论 #39093742 未加载
评论 #39092052 未加载
评论 #39091909 未加载
评论 #39092258 未加载
评论 #39093102 未加载
评论 #39093523 未加载
评论 #39092801 未加载
gurchik超过 1 年前
I recently helped save $150k per year by deleting node_modules.<p>I noticed that one of our S3 buckets had high data transfer costs, a bucket that our app downloads HTML+JS assets from when we push out a new release. I downloaded the &quot;directory&quot; of files for our latest release and saw it was mostly node_modules. I checked the code and confirmed that, yes, if this file exists in the bucket then it&#x27;ll be downloaded by the user. I wrote a quick Python script to list out each directory that had this problem, and a quick Slack message to the appropriate team later, we discovered the specific commit that was the cause, a change to our CI that inadvertently uploaded that directory when we wanted to ignore it.<p>A few months later, I checked the billing metrics, the effect was an avg of $12,500 reduction in cost for this bucket, or around $150k per year, or 4% of our bill. Not bad for one hour of work. Over the course of a quarter I reduced our bill by over $1m, or around 30% of our bill.<p>I might write a blog post explaining how to go about something like that. A lot of people are not familiar with tools like Trusted Advisor which can easily tell you if you have, for example, unused EC2 instances that can be terminated.
评论 #39094254 未加载
评论 #39092573 未加载
paxys超过 1 年前
I was thinking about this recently. I work at a large company with untold millions in AWS spend. I&#x27;m 100% confident that I could shave off a few thousands (maybe even tens or hundreds of thousands) from the bill with a little bit of effort on my side. If I go up the management chain and ask if (1) I can make this an official project and put it on the roadmap or (2) I can do this on my own time and keep some % of the savings for myself as a reward, the answer would be a very clear &quot;no&quot; to both. So overall, as an end developer I really have no incentive to work harder and ensure lower operating costs for my company, and I&#x27;m sure most developers in the industry are in the same position as me.
评论 #39094191 未加载
评论 #39094091 未加载
评论 #39094158 未加载
评论 #39096166 未加载
评论 #39094147 未加载
评论 #39094182 未加载
评论 #39094162 未加载
评论 #39094270 未加载
评论 #39094219 未加载
thefourthchime超过 1 年前
I&#x27;ve mentioned this before, in my company (big media company) I saw some S3 costs creeping up each month. I looked into it and it was a system we abandoned that was still copying files to this bucket.<p>I reached out to the team and they turned it off, it saved us $1m a year. The higher-ups rewarded me by telling me that a team should have caught this so I should meet with them now.
评论 #39091813 未加载
评论 #39094377 未加载
评论 #39091785 未加载
评论 #39091827 未加载
joshstrange超过 1 年前
One small issue I have as a developer who can spin up just about anything on AWS is this:<p>I have zero insight into the costs.<p>Yes, my company could turn that on for me but it&#x27;s rare that they do so it&#x27;s nearly impossible to know if I did something that costs a lot of money (relatively or in general) without access to the cost explorer&#x2F;billing dashboard.<p>And before &quot;well can look up what a t2.2xlarge costs and calculate it&quot;, sure. In a very contrived example I might be able to see what it costs but so many things are hidden&#x2F;hard to see in AWS. For example, I recently spun up an RDS customer on my own AWS account. After testing for a while I decided it wasn&#x27;t what I wanted and I deleted the cluster. Fast forward a month and my bill is well over what I expected (Like $30, no it&#x27;s not a ton of money but it&#x27;s my personal account and I wasn&#x27;t expecting that charge). Come to find out it created a VPC as part of the RDS cluster (I think maybe it was for the RDS proxy? Still not sure) that didn&#x27;t get deleted. I had to go chase that down and even that process wasn&#x27;t easy. I had to make sure that it wasn&#x27;t be used by anything else and then delete other things that were created when I made the RDS cluster before I could remove the VPC.<p>I was only able to do the above because I had access to the billing info. I would have left that VPC indefinitely on my work&#x27;s AWS account by accident and been none the wiser.<p>I&#x27;m more than happy to take costs into account but without access to what things are actually costing us I can&#x27;t help that much. Mostly because I need to know the costs to know what&#x27;s worth optimizing. Sure I know I could improve X feature but if that costs us pennies a day (or month sometimes) then it&#x27;s not worth it. Similarly if I know feature&#x2F;infra Y is costing $XX,000&#x2F;mo then I know I should rethink or investigate if that&#x27;s correct&#x2F;worth it.
评论 #39092386 未加载
评论 #39093244 未加载
cddotdotslash超过 1 年前
I’m convinced that once a company reaches ~$10m&#x2F;year in AWS spend it becomes entirely reasonable to hire an in-house engineer whose sole job is to find cost savings opportunities. Literally a “find unused stuff and turn it off” engineer.
评论 #39093298 未加载
nathanwallace超过 1 年前
Readers may find Steampipe&#x27;s [1] AWS Thrifty Mod [2] useful. It will automatically scan multiple accounts and regions for 50 cost saving opportunities - many of which are looking for over-provisioned or unused resources. For example, it&#x27;s crazy how much you can save by doing things like just converting your EBS volumes to the newer gp3 type. Combine with Flowpipe [3] to automate checks and actions. It&#x27;s all open source and extensible.<p>1 - <a href="https:&#x2F;&#x2F;github.com&#x2F;turbot&#x2F;steampipe">https:&#x2F;&#x2F;github.com&#x2F;turbot&#x2F;steampipe</a> 2 - <a href="https:&#x2F;&#x2F;github.com&#x2F;turbot&#x2F;steampipe-mod-aws-thrifty">https:&#x2F;&#x2F;github.com&#x2F;turbot&#x2F;steampipe-mod-aws-thrifty</a> 3 - <a href="https:&#x2F;&#x2F;github.com&#x2F;turbot&#x2F;flowpipe">https:&#x2F;&#x2F;github.com&#x2F;turbot&#x2F;flowpipe</a>
latchkey超过 1 年前
It is interesting to note that the author works at VPBank, which is one of the larger Vietnamese banks. Saving $150k per year on an AWS bill, is really nothing to them.<p>The fact that they even outsource their compute to AWS is kind of surprising when they could just fill up their existing data centers (like VNTT <a href="https:&#x2F;&#x2F;vntt.com.vn&#x2F;" rel="nofollow">https:&#x2F;&#x2F;vntt.com.vn&#x2F;</a>) with equipment, and save a whole lot more money.
评论 #39092529 未加载
ary超过 1 年前
&gt; The best optimization is simply shutting things off<p>This is the way.<p>A similar idea has been bouncing around in my mind for a while now. An ideal, turnkey system would do the following:<p>- Execute via Lambda (serverless).<p>- Support automated startup and shutdown of various AWS resources on a schedule influenced by specially formatted tags.<p>- Enable resources to be brought back up out of schedule when demand dictates.<p>- Operate as a TCP&#x2F;HTTP proxy that can delay clients so that a given service can be started when it is dormant or, even better, the service isn&#x27;t serverless but you want it to be. This can&#x27;t work for everything, but perhaps enough things such that the need to run always on services is reduced.<p>Cloud Custodian [1] can purportedly do some of this, but I&#x27;ve been reluctant to learn yet another YAML-based DSL to use it.<p>So this is my &quot;make things designed to be always-on serverless instead&quot; project and the work AWS has done to make Java apps function on Lambda keeps me thinking about the potential to take things that 1) have a relatively long startup time and 2) are designed to be long running service loops, and find a way to force them into the serverless execution model.<p>[1] <a href="https:&#x2F;&#x2F;cloudcustodian.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;cloudcustodian.io&#x2F;</a>
评论 #39093144 未加载
评论 #39094916 未加载
ejs超过 1 年前
This is especially easy if you can shutdown environments that are only used for dev&#x2F;staging tasks. With 168 hours in a week - how many hours do those things <i>need</i> to be running? I run a little tool for Heroku to make it easy to do this kinda thing.
评论 #39092123 未加载
null3cksor超过 1 年前
Couple of years ago I saved about 14 mn of revenue per year for my company. I got a 250$ bonus for it.
评论 #39092290 未加载
评论 #39092810 未加载
评论 #39096283 未加载
EvanAnderson超过 1 年前
I worked adjacent some telecom consultants in the 90s whose income was solely driven by a percentage of cost savings they could trim from telephone bills. Seemed like a very brash business model but they clearly knew there was gold to be mined.<p>I keep thinking I should be doing &quot;cloud optimization&quot; work and being compensated this way. Slicing and dicing output from usage&#x2F;billing APIs and providing an &quot;optimized spend&quot; probably has the potential for a lot of low hanging fruit.
评论 #39097960 未加载
评论 #39093377 未加载
评论 #39092365 未加载
partiallypro超过 1 年前
I saved my previous company $4000-5000&#x2F;mo on AWS billing just by auditing the AWS account and turning off unused machines that that old devs has spun up and deleting hard drives after backing them up to S3, just in case. No one had really even bothered looking at it for years and I did it in my &quot;free time&quot; at work without being tasked with it.<p>Ironically, I asked for a raise a year later and was denied, despite single handedly saving the company nearly $50000&#x2F;yr. The raise I asked for wasn&#x27;t close the cost savings I had brought. I left the company shortly after.<p>I saw someone else have a similar experience here and a comment to it was saying rewarding this produces a bad incentive...well, honestly why would I have even bothered cutting costs if I felt I wouldn&#x27;t be rewarded? Not rewarding it just makes me half regret doing it at all.
JohnMakin超过 1 年前
Looking forward to the kubernetes one - Most kubernetes clusters are designed for high availability, not necessarily for being able to quickly spin up&#x2F;down and there’s often a lot of hidden complexity there (at least on aws).
评论 #39092358 未加载
TrianguloY超过 1 年前
&gt; You go home. So you shut things down.<p>Sorry for the rant, but this is usually wrong. The amount of people that just keeps their computer on is noticeable. And when I ask it&#x27;s usually &quot;just to avoid having to wait&quot; or &quot;I&#x27;ve always done that&quot;.<p>I personally always hibernate my computer. When I turn it off it takes more time, but I&#x27;m already on the other side of the building so I don&#x27;t care. When I turn it on it takes basically the same amount of time, and it is exactly as I left it. People keep the computer on just because convenience...and I don&#x27;t think it&#x27;s a good thing.
评论 #39092681 未加载
评论 #39092478 未加载
评论 #39100236 未加载
danfritz超过 1 年前
Cut costs down to 90% by going serverless and run everything on lambdas?<p>Nothing keeps humming if it&#x27;s not being used
评论 #39092347 未加载
评论 #39092387 未加载
评论 #39092457 未加载
评论 #39092400 未加载
8organicbits超过 1 年前
You should always calculate if you&#x27;re actually going to see cost savings. Counterintuitively, running for fewer hours can increase your bill if it causes you to switch to on-demand pricing [1]. There&#x27;s a break even point you need to get past.<p>[1] <a href="https:&#x2F;&#x2F;alexsci.com&#x2F;blog&#x2F;modeling-on-demand-pricing&#x2F;" rel="nofollow">https:&#x2F;&#x2F;alexsci.com&#x2F;blog&#x2F;modeling-on-demand-pricing&#x2F;</a>
whummer超过 1 年前
To give this a slightly different spin:<p>--&gt; &quot;The best optimization is simply not spinning things up!&quot;<p>At least for local development and testing, as made possible by LocalStack (<a href="https:&#x2F;&#x2F;localstack.cloud" rel="nofollow">https:&#x2F;&#x2F;localstack.cloud</a>), among other local testing solutions and emulators.<p>We&#x27;ve seen so many teams fall into the trap of &quot;someone forgot to shut down dev resource X for a week and now we&#x27;ve racked up a $$$ bill on AWS&quot;.<p>What is everyone&#x27;s strategy to avoid this kind of situation? Tools like `aws-nuke` (<a href="https:&#x2F;&#x2F;github.com&#x2F;rebuy-de&#x2F;aws-nuke">https:&#x2F;&#x2F;github.com&#x2F;rebuy-de&#x2F;aws-nuke</a>) are awesome (!) to clean up unused resources, but frankly they should not be necessary in the first place...
datadrivenangel超过 1 年前
If you&#x27;re spending $425k per year on non-production AWS resources, you have an interesting setup.
评论 #39092235 未加载
评论 #39091834 未加载
评论 #39091848 未加载
kevin_nisbet超过 1 年前
I&#x27;ve for a long time set my cloud VMs to shutdown on idle. I usually use it to also justify running a much larger VM to cut down on build and test times.<p>Just set a cron to run the shutdown command with a grace period. And then if you&#x27;re working late, you just cancel the shutdown and the shutdown will be retried in a couple of hours. And have a script or command to just run the cloud API calls to boot the VM in the morning &#x2F; when needed, and the environment boots in a minute or two.<p>For other stuff I&#x27;ve been tempted to do a more complicated setup, with something like a micro-vm as a proxy, that will do the shutdown &#x2F; activation on TCP connection, but haven&#x27;t gotten around to it.
nodesocket超过 1 年前
I’ve mentioned this before, but probably one of the most egregious costs on AWS are NAT gateways and NAT bandwidth pricing. Typically I deploy one NAT gateway per AZ so looking at $99 a month just for three NAT gateway instances with zero traffic.
mkl95超过 1 年前
Sounds like a great way to find out who&#x27;s working late or is an early bird
StratusBen超过 1 年前
Disclosure: I&#x27;m CEO of <a href="https:&#x2F;&#x2F;www.vantage.sh&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.vantage.sh&#x2F;</a> -- a cloud cost observability platform. I previously worked at both AWS and DigitalOcean.<p>For people looking for how to save money on AWS - I&#x27;d [selfishly] recommend connecting up to Vantage. We profile AWS for all sorts of savings and give you the information on how much we can save prior to you paying us. It can be a good gut-check if nothing else on how well optimized you are.
评论 #39093131 未加载
brycewray超过 1 年前
In a similar vein:<p><a href="https:&#x2F;&#x2F;usefathom.com&#x2F;blog&#x2F;reduce-aws-bill" rel="nofollow">https:&#x2F;&#x2F;usefathom.com&#x2F;blog&#x2F;reduce-aws-bill</a>
tehlike超过 1 年前
Cutting down AWS cost by 90% by simply moving to hetzner.
评论 #39092837 未加载
评论 #39092426 未加载
throwitaway222超过 1 年前
It looks like this is surfacing because a lot of companies are laying off but also finding every possible way to save on costs right now.
kthejoker2超过 1 年前
When I joined my current company I quickly found our internal Azure environment was effectively unmanaged. Four weeks in I shaved nearly 55k in monthly spend just scripting out VM shutdowns and service pauses.<p>Cloud revenue in most large companies is at least 25$, maybe up to 40%, pure developer waste because nobody upstairs knows the difference.
hkt超过 1 年前
Small fry, I saved a public sector body $1m&#x2F;year by doing this and rightsizing the kubernetes hosts. :D
from-nibly超过 1 年前
Reacting to events to install security defaults (or any kind of defaults) sounds really error prone. Are people running AWS where devs just click buttons in aws and spin up random stuff? I thought we all decided that was dumb and switched to gitops&#x2F;terraform?<p>Did I miss a new trend or something?
bornfreddy超过 1 年前
Does anyone have a good experience with tools &#x2F; services that track and analyze cloud usage? We don&#x27;t use any, but could benefit from better visibility in spending patterns.
octopoc超过 1 年前
Here&#x27;s a startup idea: a profiler for infrastructure-as-code that shows how much each line of code is costing per month, instead of showing where the CPU spends most of its time.
评论 #39094201 未加载
评论 #39094465 未加载
TheIronMark超过 1 年前
I did this at a previous employer. I leveraged a Lambda functions and tags applied to instances to determine when they should be and when they should be off.
davidgerard超过 1 年前
We do this. It was a bit faffy to set up, but having dev and staging shut down at night and only be started as needed that day saved us a fortune.
sremani超过 1 年前
Cloud optimization is the next Kubernetes.
评论 #39091809 未加载
ralfcheung超过 1 年前
meanwhile, I&#x27;m saving the company 7-digit&#x2F;year from Google.
zikduruqe超过 1 年前
Shit... I reduced our spend on one AWS account from $270K a month to $75K a month.
评论 #39092286 未加载
gnarlouse超过 1 年前
“Have you tried turning it off and then turning it back on again?”