> CTEs won’t always be quite as performant as optimizing your SQL to be as concise as possible. In most cases I have seen performance differences smaller than a 2X difference, this tradeoff for readability is a nobrainer as far as I’m concerned. And with time the Postgres optimizer should continue to get better about such performance.<p>So, my knowledge here might be out of date, but this behavior isn't "the optimizer isn't smart enough": using a CTE in PostgreSQL causes an explicit boundary in the optimizer which prevents some optimizations from being performed. In some cases you really need/want this behavior, and it might increase the performance of your query; in other cases, this is the last thing you would want. People keep asking for optimizer hints, which PostgreSQL refuses to add, and yet they leave this quirk in explicitly as an escape hatch.
The good thing here is that other databases like MySQL, TiDB, SQLite etc also support this. so this. And even recursive CTEs are well supported.<p><a href="https://modern-sql.com/caniuse/with_(non-recursive,_top_level)" rel="nofollow">https://modern-sql.com/caniuse/with_(non-recursive,_top_leve...</a>
I sure do use them for many years now, my girlfriend now uses them and loves them, my topcoder friend eventually started using them 10 years ago.<p>So, no, we are using them , but y’all know what ? Unless recursive they are really nothing so special…
Used judiciously, a macro system in front of your SQL is often a good approach here. You gain the "linguistically common subtable" capabilities of a CTE, the ability to name constants, and the ability to name/parameterize common sub-expressions, even in databases not supporting CTEs. With LSP support being what it is, you can even whip up editor integration for your new language in a day or less.<p>Compared to CTEs (assuming you're not using recursion and don't _want_ the materialization), that extra syntactic sugar helps with readability, and the "macro" nature of the thing ensures you don't have any slowdowns from running the thing due to a different query plan.<p>Downsides include the ability for juniors to turn your SQL into an awful mess, the fact that you'll still occasionally _want_ materialized CTEs for performance and have to write them anyway, plus if you implement it wrong you'll have runtime overhead and very few introspection capabilities (the simplest version I've seen that's decent to work with is creating SQL files as artifacts from the templates using the build system, obviously depending on how much of a rube goldberg your particular builds are).
(2013)<p>It was also discussed around that time, 78 comments: <a href="https://news.ycombinator.com/item?id=7023907">https://news.ycombinator.com/item?id=7023907</a>
> CTEs won’t always be quite as performant as optimizing your SQL to be as concise as possible. In most cases I have seen performance differences smaller than a 2X difference, this tradeoff for readability is a nobrainer as far as I’m concerned.<p>This is the key trade-off you need to keep in the back of your head. Pre-mature performance optimization is the root of all evil; prefer readability first. But the fact remains that you should be setting timeouts and other time budgets, tracing calls from API through to the database, seeing what's taking the most time. If you get to the point where you need to optimize a CTE-based query, be prepared to rewrite it.
WITH clauses are a somewhat recent feature, at least on PostgreSQL. If you learned SQL before they became commonly available, it's unsurprising if you tend to think of other SQL techniques first.
>The general idea is that it allows you to create something somewhat equivilant to a view that only exists during that transaction.<p>I can't talk for Postgres but on MS SQL it's definitely not like a view.<p>I had a CTE with a filter condition in the where close but got a conversion error in the later use for a value I already filtered out in the CTE.<p>So it may look like some kind of temporary view definition but the actual code the optimizer generates may behave unexpected.
I use CTEs all the time, find them way more readable than subqueries. I've refactored a few complex queries to use them and had the reaction from the original author of "wtf is this crap?". It does take some getting used to ;)
(I'm not a dev)<p>This feels to me like a "in other news, water is wet" kind of story. Maybe it's just me, but based on the other comments here CTEs seem to be common knowledge. I went on a SQL trial by fire this year after landing in a role that required massive amounts of weird querying for ad-hoc reporting purposes. Before this my experience and knowledge stopped at different join types.<p>I naturally discovered CTEs over time after seeking solutions to my problems on the usual online resources. For me the use is two-fold. 1/ I use it to create ad-hoc lookups on data, where the lookups are not available in the DB in the first place. And 2/ In complex queries with lots and lots of conditional joins, I can use a CTE to basically build up my temporary base table, getting all my data in one place, then my main `SELECT` becomes a lot cleaner and easier to read.<p>I've not looked at recursive CTEs yet, supposedly they can be quite handy. Not sure for what though.<p>We recently migrated from MySQL to Postgres, and the data reporting went from direct-db querying to Snowflake. And CTEs work on for me on all 3 platforms, so this does not even seem like a Postgres only thing.