Hi HN,<p>After seeing all the news about lost data recently, I need to get my arse into gear and get an automated backup set up properly.<p>I'm using a Mac, so I looked into the Time Capsule. That said, if one of the data loss scenarios is a well-written ransomeware worm, it feels like the Time Capsule is going to be just as vulnerable as my main machine.<p>What approach would you recommend to back up data, with both hard drive failure and ransomware in mind? I'm open to cloud based solutions if that actually makes more sense.
I use borg[0] to create local space efficient encrypted backups and rclone[1] to mirror the archives to Google Drive. I wrote a short script to automate it and schedule it to run every night.<p>[0] <a href="https://borgbackup.readthedocs.io/en/stable/" rel="nofollow">https://borgbackup.readthedocs.io/en/stable/</a><p>[1] <a href="https://rclone.org" rel="nofollow">https://rclone.org</a>
I'm surprised that no one mentioned Tarsnap yet, it's run by a well known HNer (cperciva): <a href="http://www.tarsnap.com/" rel="nofollow">http://www.tarsnap.com/</a><p>It's not exactly noob friendly though.
There's a common rule called the 3-2-1 rule, it states that you should:<p>- Have at least three copies of your data.<p>- Store the copies on two different media.<p>- Keep one backup copy offsite.<p>Personally, I'd recommend:<p>Copy 1: Your Mac.<p>Copy 2: A local NAS (my personal choice) or hard disk.<p>Copy 3: A remote backup, stored on a hard drive in a desk drawer at work, Backblaze, Google Drive, Amazon Cloud Drive or whatever other solution suits your needs.<p>In terms of software, I personally use rsync + ZFS/BTRFS snapshots (NAS - local, NAS2 - remote) and rclone (cloud). I haven't really used fancy solutions like Attic and Borg due to their need to write dead (i.e. not mountable without a performance penalty) data to local disk or SSH. No affordable storage that I've found offers this (rsync.net offers it but is too expensive).<p>It's getting to the point where I'm seriously considering buying an LTO6/7 tape drive though...<p>I'll also add because I haven't seen it elsewhere: <i>verify your backups</i>. A backup is pointless unless you <i>know</i> you can restore it. The best way to test this is by doing it. It should get to the point where you don't fear a restore. It shouldn't be painful. There should be no worry. It should be no more than an inconvenience. When something goes wrong, you don't want there to be even the smallest hint of doubt that there's something wrong with your process.<p>As such, I <i>strongly</i> recommend having an easily accessible backup. I'd go for a spare HDD sitting in a desk drawer at home before going for cloud backups just so that you can test it frequently.
I use Arq (<a href="https://www.arqbackup.com" rel="nofollow">https://www.arqbackup.com</a>) with Amazon Drive (unlimited data for $60/year) for this
I have a setup which works really well for my photos and videos [1][2][3][4][5]. It automatically keeps a copy of each file in 3 locations; my laptop, a Synology NAS and Google Drive / Photos.<p>[1] <a href="https://medium.com/@jmathai/introducing-elodie-your-personal-exif-based-photo-and-video-assistant-d92868f302ec" rel="nofollow">https://medium.com/@jmathai/introducing-elodie-your-personal...</a><p>[2] <a href="https://medium.com/@jmathai/understanding-my-need-for-an-automated-photo-workflow-a2ff95b46f8f#.dmwyjlc57" rel="nofollow">https://medium.com/@jmathai/understanding-my-need-for-an-aut...</a><p>[3] <a href="https://medium.com/@jmathai/my-automated-photo-workflow-using-google-photos-and-elodie-afb753b8c724" rel="nofollow">https://medium.com/@jmathai/my-automated-photo-workflow-usin...</a><p>[4] <a href="https://medium.com/@jmathai/one-year-of-using-an-automated-photo-organization-and-archiving-workflow-89cf9ad7bddf#.97qsvo3cq" rel="nofollow">https://medium.com/@jmathai/one-year-of-using-an-automated-p...</a><p>[5] <a href="https://medium.com/vantage/how-to-protect-your-photos-from-bit-rot-9d0c6998121f" rel="nofollow">https://medium.com/vantage/how-to-protect-your-photos-from-b...</a>
I used to use Crashplan which had unlimited storage and was fairly cheap(like 4$/month or something) for a family plan.<p>You might want to check it out. <a href="https://www.crashplan.com/en-us/features/" rel="nofollow">https://www.crashplan.com/en-us/features/</a><p>Also it was one of the few services that had a client that worked on Linux
Here are some options that I have experience with:<p>- Time Machine with offline disks: Since Time Machine supports multiple backup destinations, you can use a Time Capsule or hard drive that's always connected to your Mac, and also have one or more additional hard drives which you connect periodically and otherwise leave in a drawer.<p>Pros: Free, built into macOS, can browse file versions directly from many apps.<p>Cons: Needs ongoing manual intervention (i.e. plugging in the offline drives). Some reliability issues… but I've experienced the most problems backing up to my own SMB/AFP shares, so a Time Capsule might be OK.<p>- Backblaze (<a href="https://www.backblaze.com/" rel="nofollow">https://www.backblaze.com/</a>) or CrashPlan (<a href="https://www.crashplan.com/" rel="nofollow">https://www.crashplan.com/</a>): Both of these online backup services have $5/month unlimited plans, and both let you specify your own encryption key (in the form of an additional password), which isn't shared with the backup provider. Note: In my experience, Backblaze's client is much lighter on system resources/battery on Mac.<p>Pros: Inexpensive, off-site storage, low-maintenance.<p>Cons: Ongoing cost, requires trust (In theory, the client software could be sharing the encryption key with the company/the NSA/your nemesis).<p>- Arq (<a href="https://www.arqbackup.com/" rel="nofollow">https://www.arqbackup.com/</a>): Paid desktop software which can back up to many different destinations, including S3, Google Drive, or your own server via SFTP. You specify an encryption key for each destination.<p>Pros: Full control. Option to back up to another machine that you own (so no ongoing cost for hosting).<p>Cons: Up-front cost. Support is less straightforward than hosted solutions since Arq doesn't provide storage.
Most importantly: it must be the backup server that has to log into your computer to backup, and not the other way around. That way, if your computer/server is compromised, the backups are still there. If you make the error to connect to the backup server, a hacker could also log into it and delete everything.<p>I my backup server uses rsnapshot and you can only log into it with ssh + key + OTP.
I use Time Machine, Arq and Amazon Cloud Drive:<p>- I have an external HDD partitioned in half: One half is for large external files that don't change much (raw files, archived data etc); and one half is a dedicated partition for Time Machine<p>- Time Machine backs up my laptop. If I lose my computer but not my hard drive, I can get a new one and seamlessly get the computer back to exactly how it was when I last backed it up, open tabs and all<p>- I also have Arq running, attached to Amazon Cloud Drive (cheapest external storage I know of). It backs up both selected portions of my laptop's disk, as well as the external hdd's non-timemachine partition (due to how TM works you can't really back it up to the cloud[1]) to "the cloud"<p>This leaves me with:<p>- Three copies of my laptop data: in the laptop, in an external hdd and in the cloud<p>- Two copies of larger data that can't fit, in the external hdd and in the cloud. My external HDD lives at home.<p>[0] Time Machine backups up once an hour, and stores backups as a simple directory structure on disk of your entire hard drive, except using hard links to old backups to avoid duplication. It keeps the last 24 hrs of hrly backups, the last 7 days of daily backups, and then weekly backups until it runs out of room.<p>This format simply doesn't work with the kind of backup where it scans a directory to see what's changed, because it effectively looks like you're adding hundreds of gigs of data each hour.
I second borg backup, I use it on my linux/mac machines<p>for windows I use reflect backup <a href="https://www.macrium.com/products/home" rel="nofollow">https://www.macrium.com/products/home</a><p>I tried acronis backup, but the disk restore failed, absolutely horrible software. then tried reflect disk restore was very smooth.
For local bootable backup I use Mac Backup Guru, which I also wrote: <a href="https://macdaddy.io/mac-backup-software/" rel="nofollow">https://macdaddy.io/mac-backup-software/</a> It's useful because it's the only software on OS X besides Time Machine which makes versioned (incremental) backups using hardlinks.<p>For remote backup I use Arq, but I have found that to be very buggy. I'm considering switching to rclone: <a href="https://rclone.org/" rel="nofollow">https://rclone.org/</a><p>With both of those backup solutions in place I should be ready for pretty much everything.
I quite like Crashplan:<p>- very reasonably priced: I pay around £10 pcm for unltd storage for my whole family<p>- zero-knowledge encryption: I have the encryption keys, and everything is encrypted on my machine before its sent up<p>- relatively low bandwidth: only ships changed files (pretty standard tbh)<p>It's saved my bacon a few times, e.g. I've used it to rescue my sister's dissertation when she wiped her laptop thinking it was in Dropbox when it wasn't. I was amazed by how easy it was for me to rescue the file from the archive.
Have all my data on Dropbox with revisions activated, and having that backedup by Crashplan. You'll have double automated backups and 0 hassle managing it.
I have a related question. I want to take backup of certain folders to a portable USB HDD every night. Can anyone recommend any simple solution for that?<p>I don't need encryption or any extraneous features. I just need the selected directories to get mirrored to a backup location.<p>Currently, I am using SyncToy by Microsoft, but I was looking for a cross platform solution.
3-2-1 rule.<p>I would use time machine capsule and periodically (weekly?) connect an encrypted external drive and Borg backup there. Next week a second drive, third week the first one...<p>Always keep one of this drive off-site.<p>This is just one of many options how to get reasonably safe (I use an almost this one just deja-dup instead of time machine.)
I've used the following method for years and it's really simple. Get an external hard drive and partition it as needed. One for your Time Machine backup and another for data. Use Google Drive to mirror the data and use Arq as your Time Machine in the cloud.
I don't backup end-systems -- but I do have a directory with important data sync'd to several systems and the cloud using syncthing. The rest of the data I care about is in git -- everything else on the system is basically disposable.
I think Amazon Cloud Drive is $60/yr if you have Prime. You can hook up your account to your Synology NAS and have it automatically back things up as soon as you copy it over. Also Synology can encrypt it on the fly as well.
attic <<a href="https://attic-backup.org/>" rel="nofollow">https://attic-backup.org/></a> encrypts data in transit via ssh, deduplicate and encrypt data at rest. I have come to appreciate both how easy it is to restore data, and the control you have over pruning which backups are kept around. You either need to be able to install attic on the remote host, or be able to mount the file system (i.e. fuse).
Do you think Google Drive (and others) are secure enough to store personal financial data like Quicken? Sometimes all you need is an email and password to get in.
I see it's already been mentioned, but allow me to second Backblaze. I use it for hundreds of clients, and they have consistently been the most reliable of the backup services I have tested<i></i>*. Since they also have versioning, you can recover from CryptoLocker variants comparatively easily as well.<p>(Almost any cloud-based backup system can help detect Locker variants. If you notice your daily backup data set suddenly shooting up in size, time to start checking for background encryption.)<p>(°Backblaze, Carbonite, Crashplan, Mozy, Acronis)
Just stick it all in Google Drive, pay the $2/mo for 100GB or $10/mo for 1TB, done. It stores the last 100 versions of each file so ransomware shouldn't be a problem, although apparently there's no way to restore a folder at a time, you'd need to do it individually for each file...