But is the RAID handling remotely sane yet? See <a href="https://arstechnica.com/gadgets/2021/09/examining-btrfs-linuxs-perpetually-half-finished-filesystem/" rel="nofollow">https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu...</a><p>It has gems such as:<p>* It won't boot on a degraded array by default, requiring manual action to mount it<p>* It won't complain if one of the disks is stale<p>* It won't resilver automatically if a disk is re-added to the array<p>I think the first is the killer. RAID is a High Availability measure. Your system is not Available if it fails to boot.
I've never lost data with btrfs, but I have purposefully avoided anything other than basic disk configurations. I use it on my workstations because it's the default in Fedora these days and haven't had any complaints.<p>On the flip-side, I've done some of the most horrible things possible and screwed up my 30-disk ZFS array a number of times and I've never lost data. I doubt btrfs could recover from anything I've done to break my ZFS pool.<p>Overall I think it's a huge shame that it was determined that the CDDL was incompatible with the GPL, because it wasn't intentional, and we would be in a completely different spot for storage otherwise.
Glad to see Btrfs getting continual updates. It's my favorite filesystem for my personal machines (work and home PCs). The feature set is just awesome. I just hope it doesn't get completely abandoned as its development seems to have slowed significantly.<p>The only thing it's missing before I consider it full-featured is stable RAID 5/6. But it looks like that hasn't been forgotten.
Maybe I'm just cynical but I think the ship has sailed for BTRFS RAID 5/6. It's now part of the global mindshare that BTRFS RAID 5/6 == data loss, no one wants to be the guinea pig that proves it works.<p>Better to direct resources towards bcachefs or ZFS IMO.
As someone who is in the middle of pulling apart a BTRFS volume by hand (read: writing code to interpret the data structures) to try and recover it, I think being burnt enough is once.<p>No indication of any hardware issue: No recent power loss (& it's on a UPS), no SMART issues, no memory test positives. But the block tree (at least, WIP) is f*cked across all the disks (looks like two competing writers went at it) and none of the available tools can deal with it.<p>It wasn't a super exotic setup either: RAID10 with 4 disks (2 stripes), fairly full and regular snapshots/cleanup, but that's it.<p>I already converted my root to ext4 because paranoia and I'm probably going to move bulk data (what can be recovered) to ZFS.
I've been managing a raid6 ext4 array with mdadm for 10 years. Started with 4 x 4TB disks and kept adding, up to 11 disks now. It works reliably and as designed. Had a few disk failures and replaced them without issues. That's one of the nice things about mdadm vs ZFS: you can add and remove disks from the array as you see fit, rather than being forced to upgrade all disks if you want to increase the size of your array.
We run BTRFS on our work laptops and have been for a few years now. Keep in mind that we're only 4 devs, so our sample size is small, but we haven't had any issues with it so far and it's been very pleasant to use so far!<p>We published our internal doc for how we install our Arch setup with fully encrypted BTRFS if anybody is curious. Happy to answer any questions too!<p><a href="https://www.lunasec.io/docs/blog/arch-linux-installation-guide/" rel="nofollow">https://www.lunasec.io/docs/blog/arch-linux-installation-gui...</a><p>(We run Thinkpad X1 Extremes (+ the business P1 equivalents) as our dev boxes.)
I ran into a major problem with performance on btrfs when used on an extremely high write volume that filled up. Between fragmentation and full disk I couldn’t even /copy/ data off the drive at a reasonable rate.<p>It was a perfect storm of me being an idiot and heavy use. But I dunno if they have remotely resolved the issues
I use BTRFS on my laptop and in case anyone is interested in trying it, <i>please</i> be sure you know how to chroot into the system in case it won't start. Arch Linux had a problematic GRUB package pushed to stable a few months ago and it bricked people's computers by booting directly to the BIOS. Whatever you do, make sure to take notes on the configuration (if it is DIY) because there are many difference between BTRFS and EXT4, so you can't use a normal chroot guide to fix your system.
I'm curious how the benchmarks are for Btrfs on 6.1, given the improvements that (I think) landed in it: <a href="https://www.phoronix.com/news/Linux-6.1-Btrfs" rel="nofollow">https://www.phoronix.com/news/Linux-6.1-Btrfs</a>
I, as many, have a BTRFS raid 5/6 story. I once worked at a place that had double digit TB databases. A few of the servers that predated me were set up with BTRFS raid 10 as their backup strategy. Basically stop the server, snapshot the volume, backup from the snapshot. It worked well enough.<p>There came a point where we were going to have to move to a larger server as there were no more slots for additional disks. New hardware procured, provisioned, prepped, ready to roll with the exception of an actual cut over. However, c-suite got involved as they tend to do from time to time, and kept kicking the can down the road.<p>Eventually disk space was a critical issue and a junior sysadmin decided it was a good idea to tell the COO that btrfs can be converted to raid 5 on the fly. I over the phone told them that this would guarantee data loss and refused. They literally came to my office to force my hand. Some people just have to learn the hard way.<p>That story aside, I'm a big fan of BTRFS and have been using in various configurations on my desktops for years. I'm glad to see this progressing.
We never had a BTRFS event on OS drives that didn't result in total loss of the OS. We have finally removed it from the last server it was put on.
I have used btrfs for a low memory footprint raid 0 solution to span drives. Mind you for expendable datasets. It has worked really well, never had failures due to btrfs shall we say -rot.<p>Filesystem is light on requirements and works really well. Compression works great too.<p>The only thing I haven't played around with are reflinks.<p>For critical datasets I apply 3-2-1 backup strategy and use ZFS.
I read all these horror stories here for RAID config and data availability/integrity and I wonder how do the Cloud providers resolve all these issues at their scale?<p>Why bare-metal admins have to spend countless man-hours on troubleshooting these?
"This should ..." doesn't belong in a fucking file system! Take a deep breath and try again. Maybe limit the scope to something within your understanding this time.
Just 'better' (less fuckups) or it works 100% and its production ready? :)<p>... just use FreeBSD with OpenZFS for RAIDZ/RAIDZ2/RAIDZ3 pools instead.<p>... or even OpenZFS on Linux.
I haven't had any issues with BTRFS yet (2yrs, arch). I've had to rollback to a previous snapshot once (a self-inflicted mistake during an update).