TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Why is Snowflake so expensive

364 点作者 eyeball超过 2 年前

48 条评论

cs702超过 2 年前
Great article. On the surface, it&#x27;s about Snowflake. At a deeper level, the article is about the perverse incentives motivating SaaS businesses to do seemingly dumb, inefficient things and avoid seemingly obvious optimizations by default.<p>Many SaaS businesses are perfectly happy to let customers shoot themselves in the foot if it generates more revenue. The BigQuery example (presently, by default, `select * from table limit 10` obediently scans the entire table at <i>your</i> expense!) is spot-on.<p>As the article so well puts it, every SaaS company has a vested financial interest &quot;to leave optimization gremlins in.&quot;
评论 #32555025 未加载
评论 #32556018 未加载
评论 #32553703 未加载
评论 #32554313 未加载
评论 #32553851 未加载
评论 #32553082 未加载
评论 #32554131 未加载
评论 #32556566 未加载
评论 #32553049 未加载
评论 #32558677 未加载
评论 #32553772 未加载
评论 #32553741 未加载
评论 #32559121 未加载
评论 #32552915 未加载
twawaaay超过 2 年前
Snowflake is not expensive. Snowflake is super cheap, <i>IF</i> you know what it is for and how to use it. Compared to if you had to solve the problem on your own.<p>The best way to describe Snowflake is that it is a brute force method to run complex queries without creating indexes.<p>If you have a more traditional database, you will notice you need to set up indexes to be able to get anything from it in finite time. What if you don&#x27;t know the indexes upfront? What if you want your users to be able to ask arbitrary queries and get answers before bedtime?<p>That&#x27;s what Snowflake is for. It automates using <i>ENORMOUS</i> amount of hardware to get your query executed fast, very inefficiently.<p>It is not for free though. That inefficiency will cause a lot of resources used for queries. It is meant for those few queries when your users try to get some insight into your data and you can&#x27;t predict indexes beforehand. Sometimes this is exactly what you want, like when you let your data people in to figure stuff out. Or when you have very rare functionality that allows the user to build their own queries -- which you should avoid like hell (and there are tricks to make it index pretty well) but can&#x27;t always avoid.<p>For everything else, whenever you can predict your indexes, you always want to use more traditional database that can be very efficient on queries properly supported by indexes.<p>The issue is a lot of people try to use Snowflake as a database or to support frequently executing queries of the same kind. This is bad and it will cost you.
评论 #32555758 未加载
评论 #32556413 未加载
评论 #32555829 未加载
评论 #32555268 未加载
评论 #32560806 未加载
stassajin超过 2 年前
I&#x27;m the author of the article. Didn&#x27;t expect it to blow up. Let me clarify a few points:<p>1. I like Snowflake and I think they brought several innovations to the field: Instant scale out&#x2F;up, time-travel, unstructured data query support. 2. Snowflake obviously makes innovations and performance improvements, otherwise they would not be the market leader they are. But I&#x27;m also suspecting that they make just enough performance improvements to be at par and then use the vendor lock in features to make switching hard.<p>My argument is that their rate of performance innovation has considerably gone down and DataBricks, Firebolt, and open source alternatives just seem more attractive from a cost&#x2F;performance ratio. I agree that Snowflake is still the best data-warehouse to start with if you have 100k, but not if you truly plan for a multi-year horizon and your usage expands.<p>- Redshift also brought a lot of innovation that allowed people to execute analytical queries 100x-1000x faster than any OLTP that existed out there. I&#x27;ve used Redshift for four years and they kept ignoring performance and features until Snowflake came out. All of a sudden because of competitor pressure, they put more effort into the product to maintain and gain market share. My hope is that Snowflake finds a solution to their innovator&#x27;s dilemma, since competitors are hot on their tails.<p>- Some people point out that 70% usage growth just shows that Snowflake is useful. Nobody disagrees with that. The issue is that majority of the companies don&#x27;t experience a 70% revenue growth to catch up with the growth in costs. At some point, you have to clamp down on costs, which means that you have to look for alternatives to run things more efficiently.
评论 #32556898 未加载
评论 #32559187 未加载
beoberha超过 2 年前
I disagree with the assertion that Snowflake has no incentive to improve performance. While I don’t work for Snowflake, I work for a competitor and we’re constantly looking to improve performance to make customers happy.<p>For the exact reason that the article claims Snowflake wouldn’t innovate, I’d assert that they would. If they are expensive and slow, and a competitor is faster and cheaper, eventually they will see business move to the competitor. We see it all the time.
评论 #32553257 未加载
评论 #32554253 未加载
carlineng超过 2 年前
[Disclaimer: former Snowflake employee]<p>Snowflake is not expensive because of perverse incentives, which is the primary claim of the article. It is expensive because it is a highly differentiated and very sticky product.<p>As others have mentioned, competition is the ultimate incentive to work on performance. Every dollar of Snowflake revenue is a dollar of revenue that Amazon, Google, Microsoft and Databricks are fighting for.
评论 #32553704 未加载
评论 #32556965 未加载
评论 #32556844 未加载
shrimalpreeti超过 2 年前
[Disclaimer: I work for a company that offers a Snowflake Cost Optimizer product] We’re an open-source monitoring &amp; alerting tool and many of our users were using it to set alerts on their warehousing (Snowflake) costs. The problem with Snowflake is particularly worse due to its lack of query level attribution of costs and no in-built features for monitoring or recommendations on improvements. We’re building a Snowflake Cost Optimizer (<a href="https:&#x2F;&#x2F;www.chaosgenius.io&#x2F;snowflake-cost-optimizer.html" rel="nofollow">https:&#x2F;&#x2F;www.chaosgenius.io&#x2F;snowflake-cost-optimizer.html</a>) and are hearing the same feedback from our customers as the author mentions. Snowflake is definitely coming up with features towards better cost transparency but I wonder if it’s too little too late.
评论 #32556395 未加载
mritchie712超过 2 年前
I predict[0] we&#x27;ll see more people choosing Clickhouse over Snowflake in the next 5 years. Clickhouse will get reasonably feature compatible with Snowflake and give people a better escape hatch if they want to self-host their data stack. Clickhouse, Inc is building a cloud product that abstracts away the complexity and there&#x27;s already companies like Altinity that will spin up a cluster for you in minutes.<p>0 - <a href="https:&#x2F;&#x2F;blog.luabase.com&#x2F;clickhouse-for-data-nerds&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.luabase.com&#x2F;clickhouse-for-data-nerds&#x2F;</a>
评论 #32553139 未加载
评论 #32553448 未加载
brianwawok超过 2 年前
Ran into the same exact thing at CircleCI.<p>Me: My builds are really slow<p>CircleCI: Here are a few very low effort answers<p>Me: git checkout is taking literally 60 seconds, but it takes 3 seconds locally, why?<p>CircleCI: Mumble Mumble.<p>They charge per minute, so why would they care if builds are slow? Was about a year of this getting worse and worse, till I finally cancelled the service last week and built my own server in my basement.<p>I know get 200% faster builds, and the hardware payback time is not very long (6 months of my CircleCI bill?).<p>I think it&#x27;s a huge red flag anytime the metric you care about is something that being &quot;worse&quot; makes the provider more money.
评论 #32554057 未加载
评论 #32554003 未加载
评论 #32554030 未加载
评论 #32554147 未加载
评论 #32553994 未加载
评论 #32554025 未加载
alberth超过 2 年前
This is all much simpler than the post makes it sound.<p>It&#x27;s usage-based pricing and customers are using more of it.<p>&gt; a customer that joins a year ago and spends $1 is paying out well over $1.7 a year later<p>The entire article is based on this 1.7x &quot;net dollar expansion&quot; statement.<p>After integrating Snowflake, customers have found value in using Snowflake and are using <i>more of it</i> 1 year later.<p>Since Snowflake is billed on usage, that explains the net-dollar expansion.
benjaminwootton超过 2 年前
The monthly bill does make me wince, but Snowflake of course includes all server and compute costs, no installation, initial configuration or upgrades etc. It’s genuine SaaS.<p>It’s also very simple to manage and optimise so less DBA or DevOps type manpower.<p>Then of course you can perfectly right size your instances and pay by the second for compute and by the byte for storage.<p>Expensive, but lower TCO than alternate approaches I suspect.
评论 #32552820 未加载
评论 #32552773 未加载
benreesman超过 2 年前
Alright I’ll bite finally. What do these companies do? Neither Snowflake’s front-facing website, nor the Wikipedia article, nor this post tell me why people pay all this money.<p>I know a bit about the effort involved in chucking around 100 petabyte datasets, and there are numerous niches a SaaS could fill in there, but it’s very murky from the outside.
评论 #32554271 未加载
评论 #32554680 未加载
glenjamin超过 2 年前
I don&#x27;t know if this is the case at Snowflake, but there are similar seemingly misaligned incentives with CircleCI&#x27;s build-seconds-based pricing model.<p>However, the generally accepted wisdom there was that improving performance had always led to more builds being run - and so still come out as a net-positive. This had happened a bunch of times as we upgraded CPUs or storage drivers or the version - there&#x27;d be a short term drop in direct revenue, but then it would bounce back quickly as people took advantage of being able to do more stuff in the same amount of time.<p>I&#x27;m told the revenue and finance people were pretty concerned the first time it happened though!
评论 #32555952 未加载
评论 #32555355 未加载
datadisruptor超过 2 年前
[disclaimer: comment written by one of cofounders of iomete - a YC-backed startup - active in the same market as Snowflake]<p>I think Snowflake is (still) expensive because it is a venture-backed enterprise software company and goes through a typical trajectory...<p>Story goes like this: founders are product-driven and first movers -&gt; find PMF -&gt; need VC funding -&gt; VCs only fund enterprise software ventures with 70%+ gross margins and high retention rates -&gt; product&#x2F;service gets priced to achieve these metrics -&gt; VCs happy to fund sales &amp; marketing machine needed to obtain sales growth, nobody cares about profitability until after IPO -&gt; startup is everyone’s darling until ~2 years after IPO.<p>Then: economic crisis hits, customers become more price sensitive, competition intensifies. Plus now management is exposed to quarterly pressure of financial markets to deliver on top-line and margin expectations.<p>Meanwhile a bunch of startups are building (lower priced) alternatives. Perhaps not as mature or feature-rich as Snowflake, but good enough for 80% of use cases that Snowflake covers.<p>Therefore the assertion that Snowflake is not optimizing their product sounds a bit crazy to me. It would be optimizing for short-term gain, while jeopardizing its reputation as the leader in the space. Obtaining excessive margins through excessive pricing only works under monopolistic conditions or if they had a truly distinctive product. Both are not the case imo. Also, it&#x27;s early days. Not exactly sure what Snowflake&#x27;s market share is, but I bet it is &lt; 5%.. so they haven&#x27;t locked in everyone yet...<p>I bet that Snowflake will be forced to compete &quot;also on price&quot; in the next five years because free enterprise is a powerful thing. The title of the article could be “Why Snowflake is (still) expensive but will get more affordable over the next few years”..
kjw超过 2 年前
<i>&quot;Snowflake has no incentive to push a code change that makes things 20% faster because that can correspond to 10–20% drop in short-term revenue. In a typical Innovator’s Dilemma, Snowflake prioritizes other things that generate an ever larger menu of compute options, like Snowpark and data apps built on Streamlit, that will bleed your organization dry.&quot;</i><p>This is not true. Snowflake has done just that - it has continuously improved performance resulting in reduced credit consumption and revenue from customers on a unit compute&#x2F;storage basis. And it has negatively impacted their revenues and stock price. Snowflake&#x27;s incentive is to strengthen their competitive position and to hopefully generate more long-term revenue from their customers.<p>The CFO forecasted a $97 million dollar short fall when guiding for 2022 revenue resulting from product improvements. Snowflake stock dropped immediately after.<p>See Q4 transcript -- <a href="https:&#x2F;&#x2F;www.fool.com&#x2F;earnings&#x2F;call-transcripts&#x2F;2022&#x2F;03&#x2F;02&#x2F;snowflake-inc-snow-q4-2022-earnings-call-transcrip&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.fool.com&#x2F;earnings&#x2F;call-transcripts&#x2F;2022&#x2F;03&#x2F;02&#x2F;sn...</a><p><i>&quot;Similarly, phased throughout this year, we are rolling out platform improvements within our cloud deployments. No two customers are the same, but our initial testing has shown performance improvements ranging on average from 10% to 20%. We have assumed an approximately $97 million revenue impact in our full-year forecast, but there is still uncertainty around the full impact these improvements can have. While these efforts negatively impact our revenue in the near term, over time, they lead customers to deploy more workloads to Snowflake due to the improved economics.&quot;</i><p>Also see the Bloomberg article -- <a href="https:&#x2F;&#x2F;www.bloomberg.com&#x2F;news&#x2F;articles&#x2F;2022-03-02&#x2F;snowflake-plunges-on-slowing-sales-growth-acquisition#:~:text=Fiscal%20fourth%2Dquarter%20revenue%20doubled,the%20period%20a%20year%20earlier" rel="nofollow">https:&#x2F;&#x2F;www.bloomberg.com&#x2F;news&#x2F;articles&#x2F;2022-03-02&#x2F;snowflake...</a>.<p><i>&quot;Snowflake Inc., a software company that helps businesses organize data in the cloud, dropped the most ever in a single day Thursday after projecting that annual product sales growth would slow from its previous triple-digit-percentage pace.<p>Executives said improvements to the company’s data storage and analysis products will let customers get the same results by spending less, which will hurt revenue in the short term, but attract more clients in the future.<p>“The full-year impact of that next year is quite significant,” Chief Executive Officer Frank Slootman said on a conference call Wednesday after the results were released. But “when customers see their performance per credit get cheaper, they realize they can do other things cheaper in Snowflake and they move more data into us to run more queries.”&quot;</i>
pykello超过 2 年前
(I am not affiliated with Keebo, although I had a recruiting meeting with them earlier this year)<p>FWIW, Keebo (<a href="https:&#x2F;&#x2F;keebo.ai&#x2F;" rel="nofollow">https:&#x2F;&#x2F;keebo.ai&#x2F;</a>) tries to solve this problem &amp; reduce your Snowflake bill by using Data Learning techniques. It can be configured to return exact results or approximate results.
评论 #32554807 未加载
georgewfraser超过 2 年前
The core claim of this article, that Snowflake doesn&#x27;t implement optimizations that would reduce usage, is not true. Search optimized tables, partitioned tables, and per-second billing are all counterexamples.
jjfoooo4超过 2 年前
This is a kind of poor engineering writing in which the author finds a product to not be tailored to his precise tastes and concludes it is because the company is user hostile and&#x2F;or doomed.<p>The bit about Snowflake not being incentivized to care about costs are trivially untrue. The rest of the article perceives trade offs as simple feature gaps.<p>For example, Snowflake gives the user more latitude to distribute workloads among “warehouses” than other offerings. With poor distribution the author will experience the workload provisioning issues he describes.
ramesh31超过 2 年前
I&#x27;m of the mind that Snowflake and Databricks are losing their value prop now that Delta Lake is open source and Iceberg is maturing. What&#x27;s to stop me from rolling my own Spark clusters and just using one of those? Is anyone doing this?
评论 #32552826 未加载
评论 #32552853 未加载
mejakethomas超过 2 年前
It&#x27;s not expensive.<p>What it can do, successfully, with three engineers was previously impossible with dozens.<p>What IS expensive is not being careful with it.
评论 #32557209 未加载
tommyphongs超过 2 年前
The article I doesn&#x27;t have exprience with Snowflake but with Cloudera&#x27;s tech stack on on-primise infrastructure. Both Cloudera and Snowflake use same approach: Separating computing and storage with main purpose: trade-of performance for scalability, easily maintaining without knowledge about user data, thus easily selling the solution to a wide range of customers without care about customer cost( maybe this also of them purpose). In my experience with Cloudera&#x27;s tech stack, it become very complexity bruce-forced system, we need install HDFS for store data( storage layer), and Hive ( basically use Mysql to keep mapping between table and the hdfs file of that table)metadata store to keep HDFS&#x27;s metadata, Impala to query engine( computing layer). Because computing layer don&#x27;t know much about how data are organized, It is very limited when we want optimise our system, query like &#x27;select * from TABLE limit 1&#x27; lead to scan overall data on many of hdfs file, and because Impala is memory computing engine, scan all table data lead to memory exceed, and because that, DA can&#x27;t use sampling data to quickly manipulate with our data. Everything leads to the hell, and because many of things can effect to our system: HDFS, Impala, Hive metadata store, etc... so very hard to fix problem when it occurred.
cedricd超过 2 年前
I&#x27;m glad the author also points out how customer (mis)use can blow up data warehouse costs too. No matter how efficient Snowflake could get, using the warehouse too much or with unnecessary queries will ultimately have a larger impact.<p>The trend in the data space currently is for usage to increase -- as more companies adopt dbt they&#x27;re running more and more prebuilt (materialized views) queries on a scheduled basis, rather than on demand. This is overall a good thing in that data is becoming easier to manage and use, but it does come at an increase in warehousing costs.<p>I think eventually the pendulum will swing back to tools that help optimize warehouse usage, as long as they allow for the same increase in productivity as dbt (disclosure - I work for one such company)
awinder超过 2 年前
I think the main metric that this is built on may be too coarse to derive the meaning that the article is. There’s conjecture that what’s driving this is more querying over the same dataset (more streamlit dashboards) but it could just as easily be expanding usage inside of companies. That’s what’s going on at my company right now, more teams using snowflake, more data being pushed in to replace existing workflows, etc.<p>I’m also not sure I understand the dig at streamlit dashboards. If you’re running hardware and introduce new read workflows, eventually you’ll need more read replicas and you’ll pay more for it. Maybe you can argue that snowflake is doing this at a higher cost but the metric data is not available in the sources to make that claim.
falcolas超过 2 年前
Snowflake is a bit generic to easily find - and the article has no hyperlinks - anybody have a one sentence summary?<p>EDIT: There it is: <a href="https:&#x2F;&#x2F;www.snowflake.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.snowflake.com&#x2F;</a><p>Data warehousing, basically.
评论 #32552547 未加载
flyinglizard超过 2 年前
Where does all this data go? It&#x27;s processed and then what? Sent to decision makers? Used to run automated processes?<p>I&#x27;m genuinely curious and would appreciate anyone who could show a real life example of this kind of pipeline where data is accumulated, then processed, then turned into revenue at the other end.<p>I&#x27;ve implemented systems that do this but my experience is that accumulating data is (too) easy, processing it in a meaningful way is slightly more challenging but ultimately driving positive business processes according to this data, which require a lot of friction with employees (training, procedures, maintenance, support) is the most difficult part.
评论 #32553635 未加载
评论 #32554950 未加载
buremba超过 2 年前
I believe they need to focus on the performance at least nowadays because both Databricks &amp; BigQuery are also great products and they push Snowflake in terms of feature-parity and performance.<p>That being said, Snowflake is also pushing for the marketplace model where you publish your app natively to move your code where the customers environment is. If they become successful, the performance might not be the one of the incentives for the companies to go with Snowflake and the switching cost might be higher as companies will move more of their business logic embedded in the system.
epberry超过 2 年前
&gt; Not providing observability to monitor and reduce costs<p>Vantage just launched this - <a href="https:&#x2F;&#x2F;www.vantage.sh&#x2F;blog&#x2F;vantage-launches-snowflake-support" rel="nofollow">https:&#x2F;&#x2F;www.vantage.sh&#x2F;blog&#x2F;vantage-launches-snowflake-suppo...</a>. The problems the author describes are almost exactly what we heard from customers:<p>- list of users&#x2F;queries that are the most expensive<p>- alerts and notifications for costs<p>- query timeout. Not something a third party can do but there is an interesting &#x27;query tagging&#x27; feature for snowflake which Vantage supports.
toto444超过 2 年前
The competition is tough in the data warehousing industry, if Snowflake is expensive people will know. Current customers may not leave but it&#x27;s going to be harder for them to get new customers.
评论 #32553223 未加载
YouWhy超过 2 年前
I often analyze tools as reduction from the space of problems × resources to the space of outcomes.<p>Let&#x27;s consider Snowflake in this paradigm<p>- Problems: analytics on data that is not laid out in a way that&#x27;s directly accessible for analysts.<p>- Resources: SQL analysts, few or no competent data engineers, spare cash<p>- Outcomes: run analytics at an industrial scale without requiring competent engineers or DevOps.<p>Since Snowflake&#x27;s optimal client gets very easily locked in, it follows up that saving said client&#x27;s money is not something even the client would care about
teej超过 2 年前
From what I can tell, the author is incorrect about the example given in &quot;Optimizer gremlins&quot;. I tested an example on my own data and micro-partition pruning was active.<p>The issue with dbt models in Snowflake is that if you ever perform a full-refresh and don&#x27;t sort it, you ruin any natural clustering that arises from an incremental model. I&#x27;ve run into this issue many times. Auto-clustering gets too expensive at scale and Snowflake doesn&#x27;t give you much guidance on alternatives.
darksaints超过 2 年前
&gt; We have 5–6 very good open-source data warehouse alternatives. We have Redshift, DataBricks, Firebolt, BigQuery, and likely a few other enterprise offerings, yet it is surprising how little training most companies have in negotiating and re-negotiating vendor contracts or in pushing for heavily discounted pricing.<p>Small nit: Redshift isn&#x27;t open source. I would also add Clickhouse, Citus, and TimescaleDB as majorly capable open source technologies with commercial offerings in this space.
jmacd超过 2 年前
Retrospectively, this is very similar to how most SaaS behaved when per user per month billing was first introduced. There were almost never any actual limits on the number of users you could add to the software, but you purchased a license for a certain number. Occasionally your account would be audited and you would be billed of the overage. It was always a significant penalty. The same was true for CPU based licenses for things like IIS, SQL Service, Oracle, etc.
KingOfCoders超过 2 年前
I have no Snowflake experience, but some limited BigQuery experience. And it&#x27;s very easy for a small company to get to $100k&#x2F;year bills without massive data.
评论 #32553454 未加载
评论 #32556960 未加载
评论 #32553204 未加载
0xbadcafebee超过 2 年前
&gt; Snowflake has no incentive to push a code change that makes things 20% faster because that can correspond to 10–20% drop in short-term revenue.<p>If they improve performance they can lower the cost to customers, which will make the product more attractive to prospective customers. But if they are already swimming in cash they may not feel the need to gain more customers.<p>Only threats prompt companies to improve things. Threat of a competitor, threat of losing all their money, threat of bad PR, threat of regulation, threat to the stock price, etc.<p>I see this every day in companies that don&#x27;t care about managing their cloud costs. They waste money like crazy because they literally don&#x27;t care if they lose money, because some exec doesn&#x27;t care, or they got enough funding until the next round, etc. A couple years later another exec asks why the CISO&#x2F;CTO is spending so much money without any ROI, and then everybody has to stop everything they&#x27;re doing to shave pennies off cloud costs.<p>Companies run by individual executives are insane. I don&#x27;t understand why people allow companies to be run this way. I think a co-op where employees could be active participants in the running of the company would allow for more sane decision-making.
rnk超过 2 年前
What most commentators are missing here is that Snowflake had a significant revenue reduction when they improved the efficiency of their product, ie they could do more with less customer cost, less cpu use. This is similar to AWS lowering prices for many things steadily over time. Snowflake did this knowing that they would get less revenue, they would have less growth, and I suspect they also knew it would cause their stock price to go down. Here&#x27;s an article on it from March, <a href="https:&#x2F;&#x2F;www.yahoo.com&#x2F;now&#x2F;snowflake-plunges-revenue-growth-outlook-223229218.html" rel="nofollow">https:&#x2F;&#x2F;www.yahoo.com&#x2F;now&#x2F;snowflake-plunges-revenue-growth-o...</a>.<p>Certainly snowflake wants to make it easier for people to spend money and solve all their problems on that platform, every company wants that. But it&#x27;s a very competitive world out there, and snowflake leaders aren&#x27;t complete idiots - they have to keep lowering their prices when they can, otherwise new people will come along and do things cheaper.
wsostt超过 2 年前
Snowflake is so expensive that Capital One has developed a toolkit for managing your instance.<p><a href="https:&#x2F;&#x2F;www.capitalone.com&#x2F;software&#x2F;solutions&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.capitalone.com&#x2F;software&#x2F;solutions&#x2F;</a>
评论 #32557308 未加载
hobs超过 2 年前
I am like 95% sure that the MAX issue he mentions is wrong - I just modified some windowing function based approaches to the one he mentions and its several OOM faster because of partition elimination.<p>Nonetheless I agree with the basic points of the article.
rsweeney21超过 2 年前
This is a great example of misaligned incentives.<p>Another example of misaligned incentives is LinkedIn. LinkedIn charges $3&#x2F;message. The more messages sent on their platform, the more money they make. They are not incentivized to help sales or recruiters target the right people. It can be a cash cow in the short term, but it creates a negative experience for your users.<p>The fact that it has worked for so long is a testament to how strong network effects are.<p>In the case of Snowflake, high switching costs will protect them for a while.
imwillofficial超过 2 年前
It’s easy to point out ways leaving in foot guns look predatory. But that’s not always the case.<p>I work for AWS in billing, and the way we calculate bills is to try to et the customer the maximum discount.<p>Things like calculating savings plan coverage from smallest to largest to maximize utilization, or turning on Reserved Instance sharing on by default within an org.<p>I would say that the seemingly gouging behavior is more often than not technical or time constraints.
manassolanki超过 2 年前
Snowflake is expensive if not monitored properly, on top of that they provide minimal observability. There are some good features like auto suspend and auto resume for cost savings but still there is scope of optimisations. For ex, they will charge you for minimum 1 minutes even if your query is running only for 2 seconds.
jwie超过 2 年前
You would think they would be saving (and charging the customer!) a bundle not enforcing constraints on their tables.<p>I’d be very interested to hear the Snowflake side of this decision, but to the customer it’s simply unforgivable to have cosmetic constraints on a database.
评论 #32553298 未加载
评论 #32552422 未加载
评论 #32552894 未加载
spullara超过 2 年前
Snowflake increases performance all the time and their customers just use more of it.
wiradikusuma超过 2 年前
So, what is Snowflake? (I assume it&#x27;s snowflake.com) From Googling it looks like Google&#x27;s BigQuery. So it&#x27;s a DB?
评论 #32580780 未加载
throw8383833jj超过 2 年前
it all comes down to the cost of switching and willingness of users to switch. the higher the cost of switching the higher you can make your product&#x27;s price. Otherwise, with an extremely low cost of switching, the cost will ultimately be driven to near zero as more and more competitors enter the landscape.
dstola超过 2 年前
&quot;optimization gremlin&quot; = dark-pattern to take as much money away from you as possile
tablespoon超过 2 年前
&gt; RevOps management<p>And now &quot;XxxOps&quot; is a meaningless buzzword.
danielodievich超过 2 年前
Interesting article. Some of it accurate. Some not.<p>&gt;&quot;Snowflake has no incentive to push a code change that makes things 20% faster because that can correspond to 10–20% drop in short-term revenue&quot; Completely untrue. There is constant optimization of scheduler, execution process, global services, and compute fabric. The famous &quot;we shipped AWS Graviton and it&#x27;s like 10%&quot; cheaper was something we did to ourselves. There is work underway to make FoundationDB faster&#x2F;more efficient too that&#x27;s totally out of this world. In short, nobody wants to burn extra CPU cycles and bill you for it.<p>&gt;&quot;Disclose Hardware Specs&quot; This isn&#x27;t hard to find if you work with Snowflake&#x27;s SE and Services, but it&#x27;s not going to give you anything. The whole POINT of Snowflake is to hide all this nonsense and make it &quot;just work&quot;. You want CPU and SSD metrics, feel free to use Databricks (many do) or whatever.<p>Now, there IS something to be said about some sort of observability into query execution as it is going. There are constant discussions on that, and some of the new upcoming features (like programmatic access to query profiler) can open that up. But yeah, Snowflake is NOT something that will open up what&#x27;s under the hood and it is super intentional<p>&gt;&quot;Not adopting benchmarks&quot; This goes around and everyone freaks out. Just profile your own work. Whatever. Nobody cares about benchmarks.<p>&gt;&quot;Optimizer gremlins&quot; Snowflake COULD do more to expose some of the internals. My job (and job of 100s of my services and technical SE colleagues) is to help customers understand what&#x27;s happening under the hood. Some of the company&#x27;s &quot;make it simple&quot; ethos COULD be a bit more open. However, much of the common things (MP pruning) can be solved by simple user education. I&#x27;ve lost count of how many customers I worked with who had 0 education in Snowflake and even like 20-30 minute intro in it made them open their eyes and go &quot;woah, I get it now&quot;. On other hand, dozens of people told me that it was amazingly easy to use without training, and it IS!<p>&gt;&quot;Improve the workload manager to increase throughput&quot; Workload manager is considerably more complex and sophisticated than this guy tells us it is. I saw an internal presentation on its internals that I asked to convert to a confluence article which thankfully happened pretty quickly and lots of people benefitted. There is cost-based scheduling that takes expected resources of queries to schedule and also considers actual resources consumed, all very frequently and for every XP. I wish that article was public but I think it will not be made one, but still, it&#x27;s definitely not FIFO.<p>&gt;&quot;Not providing observability to monitor and reduce costs&quot; This is valid feedback now and constantly what we do in services. New manageability features are coming to help with this. See CapitalOne or bunches of companies in this ecosystem.<p>&gt;&quot;What companies that use Snowflake could do better? I agree with point about education. Huge portion of people using and abusing Snowflake don&#x27;t have any formal education. Best think you can do is hire Snowflake PS or get a partner&#x2F;SI, or just take a damn class, they are REALLY good.<p>Source: 2 years in services at Snowflake with focus on perf, cost, and manageability.
msluyter超过 2 年前
Some of these complaints seem fair to me, some not as much. tl;dr -- Snowflake requires a fair bit of knowledge&#x2F;effort to use optimally.<p>I spent a number of months last year focused on lowering Snowflake spend. In the process I learned a ton about Snowflake and gained a fair amount of respect for the product. Respect as in &quot;this is really great&quot; as well as respect as in &quot;I need to be on guard here or I&#x27;m going to get hurt.&quot;<p>I think my biggest misconception at the outset was thinking of Snowflake like it&#x27;s a relational database. It&#x27;s not. Or rather, it is with a large number of caveats. Snowflake doesn&#x27;t have b-tree indexes -- rather it has &quot;clustering keys,&quot; which are sort of like coarse grained indexes that colocate data in micropartions, allowing queries to do micropartition pruning. If you have a well clustered table and you&#x27;re filtering on your clustering keys, things will be great. But if not, or, for example you have to do multi-table joins on non-clustered columns, you&#x27;ll suffer. So unless you have search optimization enabled (which costs more!), you have to retrain yourself away from &quot;oh, just add an index here or there to make things fast&quot; type of thinking you may have had working with Postgres or whatnot.<p>Regarding the author&#x27;s complaints about lack of observability, I generally found it pretty easy to analyze what was going on via the query_history table. And the built in query analyzer is quite helpful. We did add tags to our dbt runs, which was pretty easy, and I wrote a handful of queries to find like the most expensive dbt models. It wasn&#x27;t really that hard.<p>That said, dbt in particular provides a number of foot guns wrt Snowflake. Subqueries, as the author mentions, is one. We created some custom dbt macros to do things like instead of `select * from foo where x in (select * from blah)` -- if blah was small -- do a query on blah and write the query using a literal list, like `select * from foo where x in (&#x27;a&#x27;, &#x27;b&#x27;, &#x27;c&#x27;, &#x27;etc...&#x27;).<p>Another issue we discovered is that in dbt it&#x27;s trivial to create views. But we found that if views get too deeply nested, Snowflake can&#x27;t adequately do predicate pushdown. So big stacks of views on views are suboptimal.<p>Another interesting one was tests. Dbt makes it trivial to perform null or uniqueness checks against a column. We found we were spending a lot on those tests that simply were doing something like `select * from blah where col is null`. On non-cluster key columns or complex views, these were causing full table scans. We took a number of steps to mitigate those issues. (Combining queries; changing where we did these checks in the dag). The way tests are scheduled is problematic as well. One &quot;long pole&quot; test will keep your warehouse up and using credits even after the other 99.9% of the tests have completed. After some analysis we separated long pole tests from the others and put them on different warehouses.<p>I could go on and on, actually, but I think that provides a taste of some of the complexities involved. Like almost any tool, you have to really understand it to use it effectively. But it&#x27;s all too easy for, say, analysts, who may be blissfully unaware of the issues above, to write really poorly performing SQL on Snowflake.
dboreham超过 2 年前
Because someone needs a new boat?