TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

We reduced the AWS costs of our streaming data pipeline

142 pointsby cloudfalconalmost 5 years ago

9 comments

QuinnyPigalmost 5 years ago
Hmm. This looks to me like a lot of the savings were realized by moving away from managed services into a scenario where there’s more operator overhead. The AWS bill gets lower, but what about the cost of the engineering work?
评论 #23332917 未加载
评论 #23334923 未加载
评论 #23332919 未加载
评论 #23337029 未加载
评论 #23341189 未加载
评论 #23338322 未加载
评论 #23333451 未加载
merittalmost 5 years ago
I find it highly entertaining a two-year old company who was founded on the basis of helping slash cloud spending found so much waste in their own AWS spend. This is <i>not</i> an example of dogfooding, but an example of sheer incompetency and massive technical debt.<p>I&#x27;d really like to start seeing a series of blog posts from companies who are running extremely lean and efficient tech environments by utilizing cloud in an intelligent manner and avoiding the expensive and unnecessary bullshit that&#x27;s so prevalent today. The ones that can brag &quot;How we run a $4M&#x2F;yr SaaS on $40k&#x2F;yr of AWS spend!&quot; are far more interesting than &quot;How we stopped incinerating millions of VC money by simply turning off shit we didn&#x27;t need&quot;
评论 #23334546 未加载
aritraghosh007almost 5 years ago
Back when AWS started, there would be articles about the work to master scalability and performance for the modern web but as things matured, we somehow ended up in a much larger heap of literature around AWS cost optimization.
评论 #23333289 未加载
评论 #23336640 未加载
评论 #23333516 未加载
agounarisalmost 5 years ago
I am curious about the actual cost in $! Managing your own kafka or observability infra is expensive, you need a team to do this.<p>A 67% reduction doesn&#x27;t say the whole truth. They have more services to manage now, which means they need more people and more time to do this.<p>Saving 10k from your AWS bill by hiring 2 more engineers is not cost effective.
评论 #23335249 未加载
throwaway888abcalmost 5 years ago
&quot;Eliminate unused EC2 instances&quot; -27% of cost<p>Haha, so cleaned the internal IT &#x2F; DevOps mess and call it a day and than blog post it
评论 #23332652 未加载
anthonysarkisalmost 5 years ago
It seems reasonable to make some of these cost comparisons more visible.<p>ie If working on a new product or feature to understand upfront &quot;this managed service is x% more then more bare bones&quot; etc.<p>essentially turning an alchemy into a science
评论 #23335113 未加载
tyingqalmost 5 years ago
The initial pie chart seems to indicate that either AWS glue is significantly overpriced, or that they were doing something wrong.
评论 #23337061 未加载
theatrainealmost 5 years ago
Interesting idea. Does anyone do this for Azure?
dirtydroogalmost 5 years ago
We went through a similar process with GCP, which was annoying since GCP was sold as being cheaper than AWS.