I have a use case that seems to me that it should be somewhat common but I can't find any FOSS software that does it. I want to automatically assemble an arbitrary set of physical drives into a virtual drive such that all data is stored on at least 2 drives to protect against drive failures.<p>RAID 1 would seem to be able to do this, but this only works with two drives of the same size (or you sacrifice extra capacity on the larger drive). There are combined schemes like RAID 10 or other raid levels, but these all have requirements on the size and arrangement of drives.<p>I want something where:
- I can just throw an arbitrary set of drives of varying sizes at it and it figures out a way to store all data on at least two, three, or however many drives I set (ideally on a per-file basis)
- It should also support easily adding and removing drives to add capacity or remove damaged drives
- It should automatically reduplicate anything that was stored on a drive when it fails, if possible, and it should scream errors at me if it can't due to the array being too full
- Lastly, I need to be able to mount it as a filesystem (at least over a network)<p>My use case is a personal nas.<p>What tools can do this? I have yet to find any. Since stored data can be important, the tool being reputable and mature is a major factor.
There are common RAID implementations that work pretty much how you want. The RAID spec only says that in RAID 1, any piece of data must be stored on two disks. Everything past that is implementation dependent, including limitations on disk sizes or excess copies in arrays with more than two disks.<p>In particular, btrfs's RAID 1 works fine with disks of different sizes so long as it's physically possible to get every piece of data on two disks (obviously an array made of 1x2TB and 1x1TB cannot possibly have a RAID 1 volume of more than 1TB, but btrfs will happily present a 2TB RAID 1 volume on top of 2x1TB and 1x2TB). Btrfs also lets you add or remove disks at any time, even while the filesystem is in use (you do have to wait for it to copy data around to other disks before you physically detach a removed drive). I believe ZFS mirrors work similarly, but I'm less familiar with them.<p>You should also consider using raid5/6 (raidz1/raidz2 for zfs) instead of raid1. They can still survive disk failures without data loss, but you can get your storage efficiency better than 50% with them.<p>I'm not aware of anything that lets you pick the raid level per-file though.