BorgBackup: Deduplicating archiver with compression and encryption

If you prefer a similar approach, but as a single compiled binary, there&#x27;s Restic: <a href="https:&#x2F;&#x2F;github.com&#x2F;restic&#x2F;restic">https:&#x2F;&#x2F;github.com&#x2F;restic&#x2F;restic</a><p>Update: yet <i>another</i> take on basically the same approach, also as a self-contained binary: <a href="https:&#x2F;&#x2F;github.com&#x2F;kopia&#x2F;kopia">https:&#x2F;&#x2F;github.com&#x2F;kopia&#x2F;kopia</a>

I&#x27;ve been using Borg for years. It&#x27;s great! The deduplication feature allows me to take a &quot;full&quot; backup of my work station <i>hourly</i>. Taking frequent backups like this has already saved my bacon a number times in cases where I accidentally mangled&#x2F;deleted a file I didn&#x27;t mean to touch.<p>I recently stumbled upon the release notes for the (WIP) v2: <a href="https:&#x2F;&#x2F;www.borgbackup.org&#x2F;releases&#x2F;borg-2.0.html" rel="nofollow">https:&#x2F;&#x2F;www.borgbackup.org&#x2F;releases&#x2F;borg-2.0.html</a>. Seems to address quite a few of the pain points of v1.

rsync.net has a special discount when you use borg and &quot;you&#x27;re an expert&quot; <a href="https:&#x2F;&#x2F;www.rsync.net&#x2F;products&#x2F;borg.html" rel="nofollow">https:&#x2F;&#x2F;www.rsync.net&#x2F;products&#x2F;borg.html</a><p>We&#x27;re looking to replace our self-written borg backup scripts with <a href="https:&#x2F;&#x2F;torsion.org&#x2F;borgmatic&#x2F;" rel="nofollow">https:&#x2F;&#x2F;torsion.org&#x2F;borgmatic&#x2F;</a> which is a wrapper around borg.

I am the author of bupstash - <a href="https:&#x2F;&#x2F;github.com&#x2F;andrewchambers&#x2F;bupstash">https:&#x2F;&#x2F;github.com&#x2F;andrewchambers&#x2F;bupstash</a> which has many advantages over borg in my biased opinion (like air gapped decryption keys and better performance). Feel free to check it out.

I tried a few backup tools and <a href="https:&#x2F;&#x2F;github.com&#x2F;andrewchambers&#x2F;bupstash">https:&#x2F;&#x2F;github.com&#x2F;andrewchambers&#x2F;bupstash</a> is my favorite by far but it&#x27;s not that well known.<p>It was pretty fast already and recently got multithread support. It has been the only thing usable for backing up a few TB in a raspberry for performance reasons.<p>Keep in mind it&#x27;s relatively new and the author does not yet recommend to use in production as the only backup solution.

Have been using Borg for many years now, it saved me several times already when I accidentally deleted stuff I realized I still needed later on. What&#x27;s great is that you can just mount your backup repository as a FUSE filesystem, Borg then gives you a directory structure containing all your backups over time. Personally I use dates to name my backups, e.g. 2022-10-11, so when I need to restore something from a specific date I just go to the appropriate folder and extract it.

I&#x27;ve been very happy with it. I switched away from Tarsnap because it was far too slow on restores. BorgBackup has slightly different security semantics, but similarly supports the notion of append-only credentials.<p>And borgbase.com has been a good place to host my backups. Really nice onboarding flow.

Related:<p><i>Deduplicating Archiver with Compression and Encryption</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27939412" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27939412</a> - July 2021 (71 comments)<p><i>BorgBackup: Deduplicating Archiver</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=21642364" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=21642364</a> - Nov 2019 (103 comments)

borg is great. we&#x27;ve been using it for the past 3 years to archive hundreds of file-level backups of servers, database dumps and VM images. average size of each borg repo is few GB but there are few outliers up to few hundreds of GB. most of backups are done daily, with 7-24 past days preserved in borg archive. borg repos are verified, copied to external disks, verified again and rotated offline each week.<p>borg replaced <a href="https:&#x2F;&#x2F;rdiff-backup.net&#x2F;" rel="nofollow">https:&#x2F;&#x2F;rdiff-backup.net&#x2F;</a> for us and gave: * nice speedup of backups&#x2F;backup tests, * decent saving in the disk space thanks to compression and deduplication, * decreased backup replication time [ borg repo tends to have much less, larger files compared to rdiff which has in its repo at least as many files as your source data; rsync likes it ].<p>to finish backups in a reasonable time we had to parallelize backup gathering [ each server &#x2F; vm goes to separate borg repo; this limits a failure domain in case of corrupted repo, but denies us benefit of deduplication on larger scale - across servers ] and borg archiving. without that - we would be a limited by a single cpu core performance [ borg is not multithreaded yet ].<p>it&#x27;s worth testing the backups - we&#x27;re doing it each day by using borg&#x27;s repo self test and by extracting few key files and checking their checksums and content... just in case.<p>echoing other comments - <a href="https:&#x2F;&#x2F;kopia.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;kopia.io&#x2F;</a> looks interesting but we have not tried it yet.

Dunno if this 1 year old comment is relevant, but to whomever it may concern, it may be useful to research backup software stability &amp; reliability: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=29210222" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=29210222</a>

Borg is very good. The V2 repository format will bring in a lot of improvements, particularly in cryptography.<p>Anyone knows when 2.0 will be out of beta, and stable?

Just started using the append only feature to prevent a potential hacker from wiping out backups that live on a remote ssh server. Combined with restricted ssh access this can be made quite secure. I also tested writing backups to my Android phone (as a backup target) using Termux and Wireguard, worked flawlessly with a bit of tuning (keeping the vpn alive)

I’ve been using Borg for the last three years to replicate on-site ZFS snapshots as Borg archives on rsync.net. My local ZFS is plaintext on LUKS volumes and the remote Borg archives are encrypted with borg.<p>Would I be better off using ZFS native encryption and replicating the encrypted stream <i>without</i> giving a key to the rsync.net side? I worry that one false move would mean I would have to do a full backup again. I think I kind of feel more comfortable having borg as a second technology in the system, too, so that my restore can’t be broken by a ZoL bug. Doing test restores with Borg is also nice because I don’t need a ZFS system to do the restore — restoring to macOS works just fine for example.<p>Funnily enough, I’ve never had to prune &#x2F; ruler function any Borg archives. I just don’t have enough churn in my data to need to do it. Everything change is there in full.<p>It works so well. Thank you everyone involved for donating your time and ideas to providing these tools and services.

What’s good for Windows 10 (NTFS) drives? I’m using the Veeam Agent free version [0] for years and no problems whatsoever but curious what are some good options<p>0, <a href="https:&#x2F;&#x2F;www.veeam.com&#x2F;agent-for-windows-community-edition.html" rel="nofollow">https:&#x2F;&#x2F;www.veeam.com&#x2F;agent-for-windows-community-edition.ht...</a>

Borg works great. I used it at a shop that had lots of servers, but didn&#x27;t have any real backups-other than when someone would remember to go swap external USB drives and hope they actually ran.<p>Set it up on a bunch of servers with a simple cron job, the initial backups went quickly, and the incrementals were really fast. Made great use of an old Dell server that wasn&#x27;t doing anything else and had lots of slow disks in it.

Maybe a bit off topic, but what is a good utility for &quot;imaging&quot; a linux system. I have a task to reprovision a system but we want to keep a complete backup of the current system so that it&#x27;s possible to restore completely as if it were never touched.<p>This is more than just data backup as we would need need to recover disk partitions&#x2F;LVM metadata, boot records, etc. as well as all the data itself.

Can someone suggest an approach to backup container environment? E.g. running inside Kubernetes.<p>As I see it: I write some kind of configuration.<p>someproject-db is a deployment which runs a postgres db. Tool should connect to this DB, issue some kind of pg_backup command, capture output, retrieve some metadata about previous backup from S3, compute difference with previous run, compress that difference and store it to S3.<p>anotherproject is a deployment which runs an sqlite db. Tool should do the same but with sqlite-specific commands.<p>yetanotherproject-data is a pvc which has attached pv. Tool should find pod which mounted this volume, exec into that pod and retrieve pv data, again find different and store it to S3.<p>Of course things should be configurable. Like store difference every 15 minutes, store complete backup every week and so on.<p>I&#x27;m fine with manual recover and with manual configuration (I just don&#x27;t want to write and test all the scripts myself).<p>What I don&#x27;t want is some kind of magic tool which will backup the entire cluster, etcd and my grandparents automatically in some magic way only for $50k&#x2F;cpu core.

I used borg for years, never had any issue. Even though there were scares of bad versions that could corrupt your archives. Never happened to my personal backups of a few hundred Gs at most.<p>But I had to switch to restic because the S3 support was too good. Off-site backups in one command, instead of having to maintain my borg repos on NAS and external disks.

Have had a look at both Borg and Restic, but even Restic which is supposed to be faster iirc, was extremely slow on my computer.<p>Been much more happy with my tries with <a href="https:&#x2F;&#x2F;kopia.io" rel="nofollow">https:&#x2F;&#x2F;kopia.io</a> which also includes an optional cross-platform GUI, in addition to the CLI.

Unlike some of the negative comments seen, I&#x27;d like to extend my gratitude to the developers for this work!<p>I&#x27;ve recently decided to review my backup scheme, and have been looking at btrfs.<p>Anyone out there doing a btrfs+borg combo? What is your scheme?<p>I&#x27;ve found this as an example, but it&#x27;s quite a bit more intense than what I need... <a href="https:&#x2F;&#x2F;mutschler.dev&#x2F;stuff&#x2F;backup&#x2F;" rel="nofollow">https:&#x2F;&#x2F;mutschler.dev&#x2F;stuff&#x2F;backup&#x2F;</a>

I recommend using borgmatic to ease the management of Borg backups.

Borg is really solid and I had to unfortunately use it to restore data a bit ago and it went very smoothly. I’m bad with testing backups so I was a bit nervous.

Absolutely love Borg, have been using it for several years. Cross-platform (macOS and Linux) backups, deduplication, encryption and compression, FUSE mounting, backing up over SSH (works great in combination with Tailscale and can be interrupted&#x2F;resumed). Pruning backups with rules is also easy, and sometimes I accidentally back up really large folders in which case Borg lets you delete it from all the backups.

Can anyone recommend a (Linux CLI) backup tool for incremental encrypted backups via SCP that allows a verification run against the current filesystem state while excluding a configured list of paths from the verification (for some frequently changing files)? I’m currently using duplicity, which is missing that last feature (without post-processing its verification output).

Any pointers on running borg or restic on a Synology NAS running DSM 7 (to backup the NAS to another destination)? The Synocommunity package for Borg generally tends to be lagging behind the official releases by months. And Docker seems to be a bit complex too (since the NAS I have directly doesn’t support it).

Can you use this for a full system backup or is it more suited towards backing up individual files?

Is Borg significantly better than rsync to a btrfs or zfs volume, and then snapshotting that volume?

Is there anything like zfs.rent for btrfs? I would like to use my filesystem’s facilities for backing rather than a whole piece of software on top, but I haven’t found any managed storage solutions yet.

I&#x27;m searching for an alternative to OneDrive for my personal files, as I&#x27;m already paying for 1TB S3 Storage I wanted to use that as Backend.<p>What tool could I use that has a windows client, android and ios?

Hmmm I like this a lot actually, but as an encrypted file repo for sensitive files&#x2F;repos themselves. This is quite clever, don&#x27;t think I&#x27;ve seen all these features together before.

I’ve been using BorgBackup since it was Attic. Been thrilled with it.

I had to setup a quick backup to s3 storage to replace an aging rsnapshot setup. I looked at Borg, but Duplicity (<a href="https:&#x2F;&#x2F;duplicity.us&#x2F;" rel="nofollow">https:&#x2F;&#x2F;duplicity.us&#x2F;</a>) was easier to configure and connect to S3.<p>For syncing S3 storage across providers, I went with rclone (<a href="https:&#x2F;&#x2F;rclone.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;rclone.org&#x2F;</a>). Note that using rclone to sync across providers (e.g. from Amazon to Wasabi) does require the files to be downloaded to the client machine and then uploaded again. Not ideal, but if you have extra bandwidth it is a convenient setup.

&gt; Easy installation on multiple platforms: Linux, macOS, BSD, ...<p>No Windows :(<p>Luckily there are capable alternatives to Borg, including Restic and Kopia.

Its pretty great, I just wished it somehow integrated with rclone, I wish I could directly backup to google drive with it.

How does this compare to rsnapshot?

I don&#x27;t understand why creators of projects like this obscure the basics. Important things you need to tell me:<p>(1) The file I download is a command-line executable. I have to chnmod +x before I can use it. Why not share that small fact?<p>(2) How do I create a simple repository *with no encryption*. Yes, I want to start there. Why do you think I want to get a PhD in your project before trying it? No, I don&#x27;t want to start by creating encryption keys; that comes later. Start with the basic, basic, basic instructions.<p>(3) Why did I export that environment variable? I never used it, did I? Not so far as I can tell. Is that basic? No, it is not.<p>(4) What does recreate do? Is it necessary? &quot;borg rcreate --help&quot; does *not* provide any help on the encryption options. It just provides an error.<p>Seriously, what is wrong with you people? Can you not imagine what it is like to come into an open source project with no experience and no knowledge of the matter whatsoever?<p>Is it machismo? Is it elitism?<p>What, in the name of God, prevents you from saying, &quot;This is my project. It does this. This is the &quot;hello world&quot; method of trying it. Here are some cool options once you have the basics.<p>It happens almost all the time. I seriously don&#x27;t know what is wrong with you.<p>Edit: I had to search <i>forever</i> to find the keyword &quot;none&quot; that you can put after the &quot;-e&quot; parameter. Found it by trial and error. It&#x27;s in the docs, but, Jesus, people, can you not imagine for a moment that someone might be testing this on <i>not</i> the latest hardware and want to see how it runs without the complexity encryption first? You absolutely blow my mind. I don&#x27;t know if you&#x27;re putting me on, here. Do you not know how to communicate?

Python source code ? No cloud native API integration ? UI?