Maybe it is just a reflection on the state of filesystems vs databases.<p>There is no fundamental difference between a database and a flat file, it is all bytes on a disk/memory in the end. So it is mostly a question of balancing the roles of the hardware, OS, and application software.<p>For example, if the reason you are using a database is that it does a particularly good job at limiting disk IO, then it may not be necessary with fast SSDs. If your reason for splitting into small files is to save RAM, it may not be necessary if you have more RAM. If you want to do to a distributed architecture, maybe your filesystem is not up to the task and you may want a database server.
HN, are you feeling okay?<p>Many top posts in the past weeks were all about using SQLite for usecases it wasnt originally designed to do, using it as a distributed database, even.<p>Now, here's a post which misses the EXACT usecase sqlite was built for, and its barely even mentioned.<p>SQLite is quite literally the best solution here. Its a file, you open it, and boom you got a database. You can explore it with sqlite3 cli or datagrip or anything you want.<p>This is like saying "lets use png for animated images" and then searching for "a format to handle uncompressed images with transparency" and not mentioning png.
I remember when I got started with webdev around 2002, a lot was built on flat files. And so did I, because that’s how my “internet mentor” did it: guestbooks, bulletin boards, mailing lists, shoutboxes, CMSs. All in flat files, except I used php and not Perl like him.
I'm trying to get away from a DB-based CMS for some company web sites. Static generators won't do for a number of reasons, so a flat-file CMS seems like a good fit.<p>Currently I'm looking at GravCMS [1] as an alternative. It's free initially, but it can become somewhat expensive with many official plugins. But it's file format is Markdown, and one can combine multiple files into a so-called modular page. It has a backend for editing, forms and e-mailing of form submissions. Seems perfect for small and mid-sized company web site.<p>Another option I considered was Kirby [2]. Its backend UI is configurable. That's nice in theory but the documentation is somewhat lacking, in my opinion. I've used the starterpack and it took me hours to find the one command to be able to add new pages. Its content format is also custom, not Markdown. Finally, it's €100 per site.<p>Also, a few days ago, I stumbled upon Typemill [3] which I will check next week.<p>[1] <a href="https://getgrav.org/" rel="nofollow">https://getgrav.org/</a><p>[2] <a href="https://getkirby.com/" rel="nofollow">https://getkirby.com/</a><p>[3] <a href="https://typemill.net/" rel="nofollow">https://typemill.net/</a>
I can‘t recommend Kirby enough:<p><a href="https://www.getkirby.com" rel="nofollow">https://www.getkirby.com</a><p>I built many of my (project) websites with it,<p><a href="https://www.humangodinterfaces.com" rel="nofollow">https://www.humangodinterfaces.com</a><p><a href="https://www.perspectives-in-play.com" rel="nofollow">https://www.perspectives-in-play.com</a><p><a href="https://www.fabianhemmert.com" rel="nofollow">https://www.fabianhemmert.com</a><p><a href="https://www.escape-team.com" rel="nofollow">https://www.escape-team.com</a><p>it‘s blazingly fast (performance-wise and in terms of building your site), super easy to customize and I never missed my database.
I originally wrote <a href="https://www.flatpress.org/" rel="nofollow">https://www.flatpress.org/</a> as an alternative to WordPress with a flat-file database. I no longer work on it, but an enthusiast user (Arvid) took over the development, so it is now actively maintained.
Database is basically just ab abstraction to help you ingest and query flat files on disk.<p>If you skip that and do your own ingestion and querying...you just have a built-in database?<p>This might have some performance shortcomings, but given maturity of caching these days, you can probably just add caching layer in front of the inefficient query engine when the needs arise.
As someone not overly backend proficient, I love the concept of flat-file. I just rebuilt my blog with another flat file CMS, literally finished it last night, and today I am reading this article and think, whether WonderCMS may have been a better choice.. but, no I won't start over.<p>The CMS I picked is PicoCMS - what flat file stuff have you used and can recommend? And probably important, how well does it scale? I cant give much input here, my blog has 21 posts so.. no ceiling in sight yet.<p>Here I made a post where I ramble a bit about blog systems, as far as I got in contact: <a href="https://riidom.eu/posts/021_en" rel="nofollow">https://riidom.eu/posts/021_en</a>
I like plaintext as I have the option to make edits with various editors in addition to the main application.<p>This can lead to easier search and manipulation.<p>I use Logseq and Noteplan, both store data in plaintext.<p>As a secondary editor to both I use Sublime Text for very fast search and replace, or diagnosing why Logseq might be having issues.<p>I’ve had issues before in other note taking / task apps and when I run into problems it is much harder to search and manipulate a database file. That’s even if the app has a sensible database format.<p>Of course depending on how you use and might need edit data you might find databases are better for you. I’ve always found it easier to work with plaintext for note taking/ task apps.
Lately I've become a pretty big believer in flat files.<p>I don't see much value at all in small software for the sake of being small, but files are something really special.<p>You can SyncThing them. You can version control them. You can browse and delete them.<p>And best of all, they're not really flat anymore. A file is a typed object that tools know how to work with(As long as you do the sane thing and use a common file type).<p>Databases are often appropriate, but in those cases, I think sqlite is the better choice more often than not, unless you've truly got a large number of users all doing things at once.
i’m using Zola static site generator for my website (after switching from Wordpress)<p><a href="https://github.com/getzola/zola" rel="nofollow">https://github.com/getzola/zola</a><p><a href="https://mishushakov.gumroad.com/l/mish-zola" rel="nofollow">https://mishushakov.gumroad.com/l/mish-zola</a><p>if you need more than that for a static blog (like Gatsby/NextJS) i’d consider you unreasonable
I use PluXml[1] for a while on my personal blog en other sites I've created. The contents is stored in XML files. To be fast, the post creation date and tags are stored directly in the filename. This hence benefits from native OS file search.<p>[1] <a href="https://github.com/pluxml/PluXml" rel="nofollow">https://github.com/pluxml/PluXml</a>
sqlite seems better here. text files need to be parsed and loaded. fsOpen has sifficient overhead to be noticable when you use it a lot. unless you're statically pregenerating the html, sqlite is a super fast format with good library support and will give you the fastest access to the disk and its still a flat file.
Why don’t more people store all the uploaded files inside BLOBS in a relational database instead of a file system? What are the downsides?<p>I see the biggest downside being that the webserver will call your PHP app which will then proxy the data through itself one time. Well, maybe you can use ReactPHP or Swoole and then it’s faster than Node.js even…<p>Also you then have more programmable options, for example around access control lists or capabilities to access stuff, caching and expiring caches.<p>How does MySQL compre to ext3 for storing terabytes of data?<p>As a side note - what do y’all think of FUSE to Amazon S3? Just map a path to it and let it do the rest. GOOFYS I think skimps on the POSIX compliance in favor of speed.
This seems like the perfect post to spruik this cool tool (not mine, but greatly appreciated by me):<p><a href="http://ff-extractor.sourceforge.net/" rel="nofollow">http://ff-extractor.sourceforge.net/</a><p>The Flat File Extractor command line utility has saved my bacon a few times in my career. If I'm dealing with legacy code, data migrations, needing to quickly perform some surgical extraction or munging into another format - ffe is the tool for me.<p>EDIT: looks like someone's forked and update the tool. <a href="https://github.com/igitur/ffe" rel="nofollow">https://github.com/igitur/ffe</a>
When I rebuilt my poetry website (using Svelte) I was really keen to get rid of any need for a database on the backend (to save a few pennies on the hosting). I ended up with a folder for all the poems (html snippets) which the frontend can fetch when user navigates to a poem's page. The metadata associated with the poems went into a json file which gets fetched on initial page load.<p>If I did the work today I'd probably store the metadata in an SQLite database running on the frontend, but I'd keep the poem copy in their own files in the folder.
Using flat files only solves half the problem, in order to truly be able to host anywhere, you also need to be able to ditch PHP and server side processing requirements. (Wonder CMS, highlighted in this article, requires PHP with a couple of mods.)
That’s why I think static site generators are the more interesting code in this space. I’m a big fan of Gatsby, but there are several other great ones as well. This allows you to host your blog on even an old school “tilde club” type site, and it works just fine.
Sometime in the late 1990's, some programmer in some forum was bragging about the speed of some program of his that was working with a 50,000 record database. I don't remember all the exact details. I replied something along the lines that I regularly load text files having more lines than that into a text editor, that it's almost instant on today's hardware.
I feel like you can achieve some pretty awesome feats with a few files and docker. Setting up a simple Mongodb is dead simple. Adding a Fast API is dead simple. Serving a small site via Flask is dead simple. And with a simple docker exec command I can dump the database to JSON. And I can even use Mongo Compass to tinker with the database and try out queries and create indices.
Flat file is more suited toward small, well defined with simple integration requirements. A lot of simple web sites and services fall into this category.<p>If there is any possible need to integrate into a larger overall system in ways that have yet to be fully imagined or defined, a DBMS might be a better option.
The article's author seems surprised that there's a flat-file forum. What if we told them that the Fossil SCM is a flat-file (sqlite3) forum, wiki, _and_ SCM all in the same flat file? (Commenting on the original article requires an account i'm not willing to set up.)
I used to run a shared webhost provider back in the early 2000s. Flat-file software was a big deal for our customers back then because MySQL and PostgreSQL databases cost actual money.
Flat file websites make sense when you have a blog and such. I don't even know why you'd use a CMS when Next.js or SvelteKit can just generate all the pages from a folder with markdown files. Just write your content in Markdown or pull your content in at the static build stage and your content is ready to go.<p>Can anyone explain how sites like Flatboard handle user input into a flat file system? How do they handle conflicts with just one file? What's the workflow and set up like — can you use serverless or do you have to have a full server?