Friendly reminder that this post is published on the PlanetScale blog, a company that sells a database SaaS. Beware of the bias.<p>I personally would argue with every single point this article makes, except scalability.
~hello everyone, author here~<p>I know posts with ThOuGhT LeaDeRshIp titles like this are usually annoying, but I thought it would be interesting to write down some of the lessons I've been gathering as I've spent more time covering and using specific databases. My background is in data science / analytics with a couple of years of more traditional full stack here and there.<p>Broadly we've seen this pattern with infrastructure in general – it's a lot easier to set up a server than it used to be, all things considered. Now obviously if you're a tiny startup, you're more comfortable outsourcing everything to Heroku, and if you're a hyperscale enterprise, you probably want more control on exactly what your database is doing.<p>The thesis here is that on the tail end (hyper scale), things are getting more under control and predictable, and developers there want the same "nice things" you get with platforms like Heroku. Elsewhere in the ecosystem, more and more parts of the stack are getting turned into "simple APIs" (Stripe for payments, Twilio for comms, etc.). And perhaps most interestingly, as serverless for compute seems kind of stuck (maybe?), it may be the case that serverless for databases – whatever that ends up meaning – is actually an easier paradigm for application developers to work with.
As a developer, I have to say that sqlite gives me the best experience.<p>Everything else pales in comparison.<p>Create a database?<p><pre><code> sqlite3 mydata.db
</code></pre>
Where is the database?<p><pre><code> In the current directory
</code></pre>
How is it structured on disk?<p><pre><code> It's a single file
</code></pre>
How do I backup the DB?<p><pre><code> cp mydata.db /my/backups/mydata.db
</code></pre>
Do I have to install a server?<p><pre><code> No
</code></pre>
Do I have to configure anything?<p><pre><code> No
</code></pre>
During setup and deployment I usually I dabble a while with the whole GRANT *.* ON localhost IDENTIFIED BY PASSWORD or something. How do I do that with sqlite?<p><pre><code> It just works
</code></pre>
Do I have to close / protect any specific ports?<p><pre><code> No, it's just a file
</code></pre>
Which field types should I use for ... ?<p><pre><code> None. It just works.</code></pre>
The author skips the first decade of database systems in the 1960s. The oldest databases were not relational! They were hierarchical or navigational.<p>Hierarchical databases were much like a filesystem, but for records instead of files.<p>Navigational databases allowed data to be linked in a network. Look up CODASYL for detail.<p>The relational database design was first proposed in the 1970s.
> <i>NoSQL databases are maturing, for sure – we’re starting to see support for transactions (on some timeframe of consistency) and generally more stability. After years of working with “flexible” databases though, it has become clearer that rigidity up front (defining a schema) can end up meaning flexibility later on.</i><p>So funny to me that NoSQL boosters have only recently understood that designing sane schemas and knowing what order your data is inserted is important for data integrity. It's like an entire generation of highly paid software devs never learned fundamental computer science principles.
Understanding a limited amount of database internals has been very useful to me. There is one aspect of using databases that you simply cannot abstract away and that is performance. If you ask your database a question in a way it is not suited to perform or that isn't supported by indexes performance is not going to be good.<p>And these performance differences are not small once your database has a decent size. And if you tables are really large it's not a question of fast or slow but fast enough or so slow it's indistinguishable from the database being down.<p>Of course to some extent you can simply throw hardware or money at the problem. This certainly works for smaller inefficiencies, but sometimes knowing the database will give you orders of magnitude better performance. Hardware and money also don't scale indefinitely.
> MySQL, MongoDB, Firebase, Spanner; there has literally never been a better time to be a database user at any level of complexity or scale. But there’s still one common thread (ha!) among them – the focus is on infrastructure, not developer experience.<p>It was my impression that everyone picked (and still picks) MySQL, MongoDB, and Firebase _because_ they were the easiest to use. It seemed like developer experience was by far the most important thing to them (compared to sane behavior initially in the case of Mongo and MySQL, some of which has since evolved).
I work at PlanetScale and this post aligns with my thoughts on <a href="https://askdbablog.wordpress.com/2018/04/09/learning-the-fundamentals/" rel="nofollow">https://askdbablog.wordpress.com/2018/04/09/learning-the-fun...</a> whether you agree or disagree on some parts.
> Database internals will eventually just not matter<p>Of course you need to know the internals of your database. If you've ever come across a project where the team treated a key/value, or document database as a relational one (probably because the query syntax looks similar), then you will know just how important database internals are.
That does make some sense, especially for databases that have been around a while. Lots of internals were written with hard drives in mind rather than SSDs, much lower amounts of memory, and so on. On the other hand, it's nice when your database works well in even a very limited or old environment.