TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

How to use Postgres for everything

289 pointsby Olshansky5 months ago

36 comments

whateveracct5 months ago
Just don&#x27;t use a single Postgres DB for everything as you scale up to 100+ engineers. You&#x27;ll inevitably get database-as-the-API.<p>Now if you have the actual technical leadership [1] to scale your systems by drawing logical and physical boundaries so that each unit has its own Postgres? Yeah Postgres for everything is solid.<p>[1] Surprisingly rare I&#x27;ve found. Lots of &quot;successful&quot; CTOs who don&#x27;t do this hard part.
评论 #42348109 未加载
评论 #42348017 未加载
评论 #42347947 未加载
评论 #42348867 未加载
评论 #42348174 未加载
评论 #42349335 未加载
评论 #42348467 未加载
评论 #42348028 未加载
jensneuse5 months ago
I absolutely love Postgres, but please allow me to say that you absolutely don&#x27;t want to expose an API generated from a database to people outside of your team. This limits you a lot in changing the way you store your data.<p>I wrote about this topic before and haven&#x27;t changed my opinion much. You don&#x27;t want to have that tight coupling: <a href="https:&#x2F;&#x2F;wundergraph.com&#x2F;blog&#x2F;six-year-graphql-recap#generated-graphql-apis:-tight-coupling-as-a-service" rel="nofollow">https:&#x2F;&#x2F;wundergraph.com&#x2F;blog&#x2F;six-year-graphql-recap#generate...</a>
评论 #42348138 未加载
评论 #42348102 未加载
tlarkworthy5 months ago
I was recently annoyed to find postgres indexes don&#x27;t support skipping [1] you also can&#x27;t have the nul character in a string (\u0000) [2]. Its great, but it has some strange WTF gaps in places.<p>[1] <a href="https:&#x2F;&#x2F;wiki.postgresql.org&#x2F;wiki&#x2F;Loose_indexscan" rel="nofollow">https:&#x2F;&#x2F;wiki.postgresql.org&#x2F;wiki&#x2F;Loose_indexscan</a><p>[2] <a href="https:&#x2F;&#x2F;stackoverflow.com&#x2F;questions&#x2F;28813409&#x2F;are-null-bytes-allowed-in-unicode-strings-in-postgresql-via-python" rel="nofollow">https:&#x2F;&#x2F;stackoverflow.com&#x2F;questions&#x2F;28813409&#x2F;are-null-bytes-...</a>
评论 #42349449 未加载
评论 #42349491 未加载
评论 #42349245 未加载
jeeybee5 months ago
PGQueuer is a lightweight job queue for Python, built entirely on PostgreSQL. It uses SKIP LOCKED for efficient and safe job processing, with a minimalist design that keeps things simple and performant.<p>If you’re already using Postgres and want a Python-native way to manage background jobs without adding extra infrastructure, PGQueuer might be worth a look: GitHub - <a href="https:&#x2F;&#x2F;github.com&#x2F;janbjorge&#x2F;pgqueuer">https:&#x2F;&#x2F;github.com&#x2F;janbjorge&#x2F;pgqueuer</a>
评论 #42349028 未加载
评论 #42348973 未加载
评论 #42349057 未加载
j12a5 months ago
Being stuck with MariaDB&#x2F;MySQL in some projects, I recently compared it to PostgreSQL and found many of these extended capabilities existing there also, including JSON, temporal tables (w&#x2F; SYSTEM VERSIONING), columnar and vector storages etc.<p>LISTEN&#x2F;NOTIFY type functionality was sort of missing but otherwise it was surprising how it is keeping up also, while little of that is probably being used by many legacy apps.
jankovicsandras5 months ago
Can you please add this shameless plug?<p><a href="https:&#x2F;&#x2F;github.com&#x2F;jankovicsandras&#x2F;plpgsql_bm25">https:&#x2F;&#x2F;github.com&#x2F;jankovicsandras&#x2F;plpgsql_bm25</a><p>Opensource BM25 search in PL&#x2F;pgSQL (for example where you can&#x27;t use Rust extensions), and hybrid search with pgvector and Reciprocal Rank Fusion.
评论 #42349357 未加载
评论 #42348761 未加载
KingOfCoders5 months ago
&quot;It&#x27;s inspired by this article from the Amazing CTO [0]&quot;<p>Getting up in the morning, seeing an article that references you, bliss!<p>[0]: <a href="https:&#x2F;&#x2F;www.amazingcto.com&#x2F;postgres-for-everything&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.amazingcto.com&#x2F;postgres-for-everything&#x2F;</a>
评论 #42348735 未加载
ianpurton5 months ago
I can see a lot of benefits of having 1 API to access a lot of functionality.<p>For example instead of integrating with a message queue I can just do an INSERT this is great. It lowers the friction.<p>Vector search is a no brainer too. Why would I have 2 databases when 1 can do it all.<p>Using Postgres to generate HTML is questionable though. I haven&#x27;t tried it but I can&#x27;t image its a viable way to create user interfaces.
评论 #42348376 未加载
bkovacev5 months ago
While we are it - are there any good resources on how to best self host a Postgres database? Any tips and tricks, best practices, docker &#x2F; no docker etc? I’m looking to self host a database server for my multiple pet projects, but I would love to get backups, optimizations and other stuff done well.
评论 #42348856 未加载
评论 #42349227 未加载
评论 #42353930 未加载
评论 #42348908 未加载
评论 #42348870 未加载
lokimedes5 months ago
Having just spent the better part of two weeks integrating Apache Age for Graph data, just to realize the project is stale and a mess, don’t take this list on face value.<p>Now hoping for better results with DGraph, but it seems that graph databases are living a precarious existence.
评论 #42351096 未加载
评论 #42348777 未加载
评论 #42348190 未加载
评论 #42348803 未加载
rdsubhas5 months ago
This is a lovely list, thank you. But what&#x27;s really missing is multi master and high availability. I&#x27;m glad to see that partitioning via sharding is covered.<p>IMHO the true limitations of RDBMS are not about usage, but scaling: Multi master across simple zones, High availability, Partitioning.<p>(IMHO it comes from ACID compliance, so I don&#x27;t know if it&#x27;s even solveable natively)
评论 #42353954 未加载
评论 #42349139 未加载
rcleveng5 months ago
I like this.<p>I strongly believe that picking some tech stack you know when you are starting out is always the right decision, until it&#x27;s not. Only then do you pick a different solution.<p>Better to move fast leveraging what you know, until you need something else.
cpursley5 months ago
Thanks for the gist mention!: <a href="https:&#x2F;&#x2F;gist.github.com&#x2F;cpursley&#x2F;c8fb81fe8a7e5df038158bdfe0f06dbb" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;cpursley&#x2F;c8fb81fe8a7e5df038158bdfe0f...</a>
rozenmd5 months ago
I&#x27;ve been running my project as a business for almost four years now and I think my Postgres DB is the only thing that I haven&#x27;t changed.<p>Easily the most stable part of the stack (100% uptime on RDS since 2021&#x2F;02&#x2F;01).
评论 #42353969 未加载
paradite5 months ago
Anyone experienced with postgres full text search?<p>I want to get something simple setup but couldn&#x27;t get it to work.<p>I want to match substrings like &quot;&#x2F;r&#x2F;chatgpt&quot; (sub reddits) in url links, but couldn&#x27;t get it to match.<p>Tried a few types of queries like phrase, plain, default, simple, english. All have some weird issues, either not matching special characters, or not matching substrings (partial match). Also I&#x27;m somewhat limited on the syntax side by what can be done with drizzle ORM.
评论 #42348845 未加载
dmezzetti5 months ago
I fully agree. Postgres has solved many of the problems that many are re-solving with GenAI related databases.<p>With txtai (<a href="https:&#x2F;&#x2F;github.com&#x2F;neuml&#x2F;txtai">https:&#x2F;&#x2F;github.com&#x2F;neuml&#x2F;txtai</a>), I&#x27;ve went all in with Postgres + pgvector. Projects can start small with a SQLite backend then switch the persistence to Postgres. With this, you get all the years of battle-tested production experience from Postgres built-in for free.
cbeach5 months ago
I love Postgres but wouldn&#x27;t want to use it for everyting.<p>Taking graph databases, for example, Postgres (via Apache AGE) stores graph data in relational tables with O(log(n)) index lookups and O(k ⋅ log(n)) traversal*<p>Whereas a true graph database like Neo4J stores graph data in adjacency lists, which means O(1) index lookups and O() traversal.<p>That&#x27;s a massive difference in traversal complexity for most graphs.<p>*k is the degree of the node (number of edges connected to a node).
评论 #42359814 未加载
Ambolia5 months ago
What about binary blobs? I hate having to store them outside of the database and that you can&#x27;t just do a dbdump&#x2F;import to clone a project.
评论 #42350235 未加载
ksynwa5 months ago
Anyone has advice for backup up a postgres database that is run for personal use? I currently have one that is used by miniflux. I wrote a script for backing it up but it is gonna fail me some day.<p>``` #!&#x2F;usr&#x2F;bin&#x2F;env sh<p>set -e<p>PGUSER=miniflux PGPASSWORD=... pg_dump -F t -h 127.0.0.1 miniflux | gzip &gt; &#x2F;backups&#x2F;miniflux_db_temp.tar.gz mv &#x2F;backups&#x2F;miniflux_db{_temp,}.tar.gz ```
评论 #42349412 未加载
评论 #42351060 未加载
评论 #42362991 未加载
lenkite5 months ago
Wonder if someone can write a SQLite for everything article too - would be far more difficult though if considering HA.
jascha_eng5 months ago
The markdown is a bit broken. But I agree with the message.<p>Created a PR to update some of the vectordb links and also fixed the markdown: <a href="https:&#x2F;&#x2F;github.com&#x2F;Olshansk&#x2F;postgres_for_everything&#x2F;pull&#x2F;1">https:&#x2F;&#x2F;github.com&#x2F;Olshansk&#x2F;postgres_for_everything&#x2F;pull&#x2F;1</a>
adamcharnock5 months ago
If anyone has seen a steams implementation in Postgres please let me know. Not just pub&#x2F;sub, but full implementation of streams with consumer groups would be very interesting to me.<p>Second best, how would you implement this in Postgres? I’m tempted to give it a go but I haven’t a fully-baked plan yet.
评论 #42348591 未加载
Havoc5 months ago
While I appreciate postgres flexibility this feels like a mistake. A bit like a person insisting Rust is best and using other languages are unnecessary.<p>You could, but it is going to force technical compromises. Even tools that can do everything aren&#x27;t the best at everything.
feverzsj5 months ago
Postgres administration is such a PITA. I&#x27;d rather use sqlite for everything.
评论 #42349006 未加载
mkaski5 months ago
Glad to see pg_render on the list! Here&#x27;s a demo site rendered entirely using just Postgres(t): <a href="https:&#x2F;&#x2F;pgrender.org" rel="nofollow">https:&#x2F;&#x2F;pgrender.org</a>
评论 #42350369 未加载
joeevans10005 months ago
Great repo! Thanks!<p>I&#x27;m wondering, any approach for bitemporal dbs, like xtdb?
评论 #42348338 未加载
wiseowise5 months ago
s&#x2F;postgres&#x2F;sqlite&#x2F;g
maxboone5 months ago
One of the problems I have with this is that I wonder if this doesn&#x27;t majorly impact blue&#x2F;green upgrades and (actual cluster, not schema) migrations.
polote5 months ago
This idea that Postgres should be used for everything really need to die in a professional context.<p>I was appointed in a company of 10 dev that did just that. All backend code was PostgreSQL functions, event queue was using Postgres, security was done with rls, frontend was using posgtraphile using graphql to expose these functions, triggers were being used to validate information on insert&#x2F;update.<p>It was a mess. Postgres is a wonderful database, use it as a database. But don&#x27;t do anything else with it.<p>Before some people come and say &quot;things were not done the right way, people didn&#x27;t know what they were doing&quot;. The dev were all fan of Postgres contributing to the projects around, there was a big review culture so people were really trying to the best.<p>The queue system was locking all the time between concurrent requests =&gt; so queue system with postgres works for a pet project<p>All the requests were 3 or 4 times longer due to fact that you have to check the rls on each row. We have also all pour API migrated now and each time the sql duration decrease by that factor ( and it is the exact same sql request ). And the db was locking all the time because of that as it feels likes rls breaks the deadlock detection Postgres algorithm<p>SQL is super verbose a language, you spend your time repeating the same line of code , it makes basic function about 100 lines long when they are 4-5 lines in nodes js<p>It is impossible to log things inside these functions to have to make sure things will work and if it doesn&#x27;t you have no way to know where the code did go through<p>You can&#x27;t make external API call, so you have to use a queue system to make any basic things there<p>There are not real lib , so everything need to be reimplemeted<p>It is absolutely not performant to code inside the db, you can&#x27;t do a map so you O(n2) code all the time<p>API were needed for the external world , so there was actually another service in front of the database for some case and a lot of logic were reimplemeted inside it<p>There was a downtime at each deployment as we had to remove all the rls and recreate them ( despite the fact that all code was in insert if not update clauses) it worked at the beginning but at some point in time it stopped working and there was no way to find why, so drop all rls and recreate them<p>It is impossible to hire dev that wants to work on that stack and be business oriented , you would attract only purely tech people that care only about doing there own technical stuff<p>We are almost out of it now after 1 year of migration work and I don&#x27;t see anything positive about this Postgres do everything culture compared to a regular node js + Postgres as a database + sqs stack<p>So to conclude, as a pet project it can be great to use Postgres like that, in a professional context you are going to kill the company with this technical choice
评论 #42349272 未加载
christophilus5 months ago
Has anyone here run Firebird in production? It seems simpler (from an ops &#x2F; sysadmin point of view).
tanin5 months ago
Great repo. I love Postgres too. Use it for everything until there&#x27;s an actual bottleneck
osigurdson5 months ago
Postgres, ClickHouse and NATS for everything
评论 #42350392 未加载
anonzzzies5 months ago
Is there a local-first postgres solution?
评论 #42348266 未加载
cynicalsecurity5 months ago
Postgres cult strikes back.
saisrirampur5 months ago
TL;DR from many comments: just don’t do it! I’m glad this is becoming mainstream, and people are realizing that Postgres for everything doesn’t work. Don’t get me wrong—I’m a huge Postgres proponent and have spent 10 years helping customers implement it. However, I’m a strong believer in using Postgres for what it’s designed for in first-place.<p>Postgres was designed as a row-based OLTP database, with over 30 years of effort dedicated to making it robust for that use case.I know there are many extensions attempting to make Postgres support other use cases, such as analytics, queues, and more. Keep in mind that these extensions are relatively recent and aim to retrofit new capabilities onto a database primarily designed for transactional workloads. It’s like adding an F1 car engine to a Toyota Camry — will that work?<p>Extensions also have many issues—they are not fully Postgres-compatible. In Citus, for example, we added support for the COPY command four years into the company, and chasing SQL coverage was a daily challenge for 10 years. Being unable to use the full capabilities of Postgres and having to work around numerous unsupported features defeats the purpose of being a Postgres extension.<p>On the other hand, you have purpose-built alternatives like ClickHouse and Snowflake for analytics, Redis for caching, and Kafka for queues. These technologies have benefited from decades of development, laser-focused on supporting specific use cases. As a result, they are robust and highly efficient for their intended purposes.<p>I often hear that these Postgres extensions are expanding the boundaries of what Postgres can do. While I partly agree, I also question the extent to which these boundaries are truly being expanded. In this era of AI, where data is growing exponentially, handling scale is critical for any technology. These boundaries will likely be broken very quickly.<p>Take queues as an example: you have a purpose-built technology like Kafka or a Postgres extension that supports queues. For an early-stage startup, adopting a less optimized Postgres-based solution <i>may</i> (not a guarantee) save a few weeks of initial CapEx costs compared to using an optimized solution like Kafka. However, 6 to 12 months later, you may find yourself back to square one when the Postgres-based queue fails to scale. At that point, migrating to a purpose-built technology becomes an arduous task—your system has grown, and now it may take months of effort and a larger team to make the switch.<p>Ultimately, this approach can cost more time and money than starting with a purpose-built solution from the beginning, which might have only required a few extra weeks of CapEx. I’ve seen this firsthand at Citus, where customers like Cloudflare and Heap eventually migrated to purpose-built databases like ClickHouse and SingleStore respectively. While these migrations happened a few years later, times have changed — data grows faster now, and the need for a purpose-built database arises much sooner. It’s also worth noting that Citus was an incredible piece of technology that required years of development before it could start making a real impact.<p>TL;DR: Please think carefully before choosing the right technology as you scale. Cramming everything into Postgres might not be the best approach for scaling your business.
评论 #42414549 未加载
29athrowaway5 months ago
Broken markdown.<p>- the entire repository is 1 file<p>- the file is titled &quot;read me&quot;<p>- but you didn&#x27;t read it (it&#x27;s not proofread)<p>why do you want me to read something you did not read?