The talk linked at the end is really interesting, even as someone who doesn't know anything about filesystems outside of the classic Unix they teach in school and reading a lot of deep discussions about tuning ZFS. (None of my ZFS users are strenuous enough to require any tuning but I can't help but learn and optimize when there are levers to pull).<p>It's interesting that it's implementing all these modern features on top of extent-based non-COW transactional btrees (though it looks like APFS is the same way). With the very database-like architecture at the low level, it would be fun to stick a SQL interface on top. You should be able to do advanced queries very efficiently without having to fall back to scanning every file in the directory like you would normally have to.<p>I'm salivating looking at the caching/tiering features. People often try to implement a tiered filesystem with ZFS L2ARC, SLOG, and now the special vdev. But none of those features were intended for that and it shows.
Bcachefs is one of the most interesting projects going on in Linux kernel land these days, all the more remarkable because it is mostly the work of one person over the last 12 years. I hope it goes upstream soon; with a bit of polish it should be ready for widespread adoption.
Does anyone know if any of these next generation filesystems can do raid6 with more flexibility than normal linux software raid?<p>Say you do a 6x4TB raid 6 array in Linux and get a 16TB virtual drive. Now if you update 4 of the drives to 8TB will still get 16TB with 4x4TB of fully unused space. But 4x4TB is itself enough to do 8TB of raid6 that would get you a total of 24TB usable space where any two drives can fail. Does any of these filesystems just do that automatically and continuously upgrade the capacity as you phase in new disks?
I didn't understand even half of that but am still excited. I have no less than 6 external HDDs and SSDs lying around and it's been a pain to make a good stable ZFS pool out of them. `bcachefs` seems like a perfect match.
I eagerly await this work stabilizing to the point of merging to mainline. I'm impressed with how much progress Kent has made, but I'm frustrated by how often he seems to say some feature is basically ready except for this bug and except for that problem. I understand great designs often come from the mind of a single inspired individual, but I hope that he's ready to accept contributions from others and that more developers see the value in helping to get this across the finish line.
More info and highlights about bcachefs:<p><a href="https://bcachefs.org/" rel="nofollow">https://bcachefs.org/</a><p>Bcachefs is a new filesystem for Linux.<p>Copy on write (COW, like zfs or btrfs), Full data and metadata checksumming, Multiple device support, Replication, Compression, Encryption, Snapshots, Caching<p>Erasure coding (not quite stable)<p>Scalable - tested to 50+ TB<p>Already working and available today
> The lock
ordering rules had become too complicated and this was getting us too many
transaction restarts, so I stole the standard technique from databases<p>What is the standard technique from databases?
Just curious about the mention of persistent data structures. Is this about the FP notion of persistent data structures (which I suspect) or on-disk data structures? Or both? :)<p>I find that (FP) persistent data structures are like a super-power in lots of ways. (And it only costs a log-N slowdown in the worst case.)