Rsync replaced with openrsync on macOS Sequoia

Looking at the sparse documentation of openrsync does not create any confidence for me that it can be an acceptable substitute for rsync.<p>In my opinion, any program that is supposed to copy files, but which is not able to make perfect copies, i.e. copies that do not lose any bit of data or metadata that was present in the original file, is just unusable garbage.<p>Unfortunately, most copying programs available in UNIX-like operating systems (and also many archiving programs) do not make perfect file copies with their default options and many of them are never able to make perfect copies, regardless what options are used.<p>I have not looked recently at the scp command of ssh, but at least until a few years ago it was not possible to make perfect file copies with scp, especially when the copies were done between different operating systems and file systems. That is why I never use scp, but only rsync over ssh.<p>Rsync is the only program that I have seen, which is able (with the right options) to make perfect file copies even between different operating systems and file systems (for instance between FreeBSD with UFS and Linux with XFS), preserving also metadata like extended file attributes, access control lists and high-precision file timestamps (some copying programs and archiving programs truncate high-precision timestamps).<p>The current documentation of openrsync does not make any guarantee that it can make complete file copies, so by default I assume that it cannot, so for now it is a program that I consider useless.<p>Beside rsync for copying, one of the few Linux archiving programs that can archive perfect file copies is bsdtar (when using the pax file format; the ancient tar and cpio file formats cannot store all modern file metadata).<p>(FYI: I always alias rsync to &#x27;&#x2F;usr&#x2F;bin&#x2F;rsync --archive --xattrs --acls --hard-links --progress --rsh=&quot;ssh -p XXX -l YYYYYYY&quot;&#x27;)<p>(With the right CLI options, &quot;cp&quot; from coreutils can make perfect file copies, but only if it has been compiled with appropriate options; some Linux distributions compile coreutils with wrong options, e.g. without extended file attributes support, in which case &quot;cp&quot; makes only partial file copies, without giving any warnings or errors.)

As a relatively new Linux user, I often find the &quot;versioning&quot; of bundled system utilities also to be a bit of a mess, for lack of a better word.<p>A classic example, at least from my experience, is `unzip`. On two of my servers (one running Debian and the other an older Ubuntu), neither of their bundled `unzip` versions can handle AES-256 encrypted ZIP files. But apparently, according to some Stack Overflow posts, some distributions have updated theirs to support it.<p>So here is what I ran into:<p>1. I couldn&#x27;t easily find an &quot;updated&quot; version of `unzip`, even though I assume it exists and is open source.<p>2. To make things more confusing, they all claim to be &quot;version 6.00&quot;, even though they obviously behave differently.<p>3. Even if I did find the right version, I&#x27;m not sure if replacing the system-bundled one is safe or a good idea.<p>So the end result is that some developer out there (probably volunteering their time) added a great feature to a widely used utility, and yet I still can’t use it. So in a sense, being a core system utility makes `unzip` harder to update than if it were just a third-party tool.<p>I get that it&#x27;s probably just as bad if not worse on Windows or macOS when it comes to system utilities. But I honestly expected Linux to handle this kind of thing better.<p>(Please feel free to correct me if I’ve misunderstood anything or if there’s a better way to approach this.)

On one hand, it&#x27;s a little annoying that openrsync doesn&#x27;t support some features that rsync does.<p>On the other hand, it&#x27;s <i>great</i> that there are multiple independent implementations of rsync now. It means that it&#x27;s actually being treated as a protocol, not just a piece of software.

Patches to mainline rsync added support for extended attributes, particularly for supporting macOS metadata.<p>Bombich &quot;Carbon Copy Cloner&quot; is a GUI app that wraps it.<p><a href="https:&#x2F;&#x2F;support.bombich.com&#x2F;hc&#x2F;en-us&#x2F;articles&#x2F;20686446501143-Credits#h_01HN3XNGRYMD91E8BQE6JJ56NF" rel="nofollow">https:&#x2F;&#x2F;support.bombich.com&#x2F;hc&#x2F;en-us&#x2F;articles&#x2F;20686446501143...</a><p>I started following Mike Bombich from his posts on macOS Server sysadmin boards; see<p><a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20140707182312&#x2F;http:&#x2F;&#x2F;static.afp548.com&#x2F;mactips&#x2F;rsync.html" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20140707182312&#x2F;http:&#x2F;&#x2F;static.afp...</a><p>Nathaniel Gray created a testing tool to verify the fidelity of backups; files with multiple streams, extended attributes and ACLs, all the good stuff... Backup Bouncer:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;n8gray&#x2F;Backup-Bouncer">https:&#x2F;&#x2F;github.com&#x2F;n8gray&#x2F;Backup-Bouncer</a><p>See also this SwiftUI app that wraps rsync, RsyncX.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;rsyncOSX&#x2F;RsyncOSX">https:&#x2F;&#x2F;github.com&#x2F;rsyncOSX&#x2F;RsyncOSX</a><p>We used to really care about this stuff, back when we were still running software from &quot;Classic&quot; macOS on top of our new UNIX systems.<p><a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20161022012615&#x2F;http:&#x2F;&#x2F;blog.plasticsfuture.org&#x2F;2006&#x2F;03&#x2F;05&#x2F;the-state-of-backup-and-cloning-tools-under-mac-os-x&#x2F;" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20161022012615&#x2F;http:&#x2F;&#x2F;blog.plast...</a>

So, anyone got a good resource on why Apple is so afraid of GPLv3? Surely this shouldn&#x27;t be a problem as long as they statically compile the executables?

I looked at openrsync when I was writing my own <a href="https:&#x2F;&#x2F;github.com&#x2F;gokrazy&#x2F;rsync">https:&#x2F;&#x2F;github.com&#x2F;gokrazy&#x2F;rsync</a> implementation (in Go!) and it’s good code :)<p>It’s a shame that openrsync is not 100% compatible with rsync — I noticed that Apple was starting to switch to openrsync because my own tests broke on macOS 15.

&gt; openrsync is written as part of the rpki-client(1) project, an RPKI validator for OpenBSD. openrsync was funded by NetNod, IIS.SE, SUNET and 6connect.<p>Could anyone suggest why these organizations would want to fund this development?<p><a href="https:&#x2F;&#x2F;github.com&#x2F;kristapsdz&#x2F;openrsync?tab=readme-ov-file#project-background">https:&#x2F;&#x2F;github.com&#x2F;kristapsdz&#x2F;openrsync?tab=readme-ov-file#p...</a>

For a while, (up to including Sequioa 15.3) both rsync_samba and rsync_openrsync were available, via &#x2F;var&#x2F;select&#x2F;rsync or the env variable CHOSEN_RSYNC.<p>One particular annoyance of openrsync is that it claimed to support the &#x2F;.&#x2F; magic path element for --relative. I sent a bug report to Apple for this about a month ago.<p>rsync_samba is gone as of Sequoia 15.4.<p>I&#x27;ve installed rsync from homebrew.

I recently ran into an issue with this because building an iOS .ipa from the command line with xcodebuild apparently ends up shelling out to call rsync to copy some files between local directories, and because I had homebrew rsync earlier in $PATH, it would end up running homebrew rsync, but xcodebuild passed an openrsync-only command line argument &quot;--extended-attributes&quot; that homebrew rsync doesn&#x27;t understand and would exit with a failure.

I continue to be happy that Apple continues to enhance and embrace the posix side of osx vs gradually stripping it away in some kind of attempt to make it more like iOS.

Just like they replaced bash with zsh. Most Big Tech firms are allergic to GPL3.

Looks like OpenBSD maintains openrsync.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;kristapsdz&#x2F;openrsync">https:&#x2F;&#x2F;github.com&#x2F;kristapsdz&#x2F;openrsync</a>

I see most people are discussing BSD VS GPL v3 and some wonder why on earth do we need more than one implementation of the rsync protocol.<p>My view is that having more than one choice is good. It is good for both people and companies that we have BSD and Linux. It is good we have both BSD and GPL.<p>Sometimes, having too many choices is bad because it leads to fragmentation, creates support and technical issues and leads to analysis paralysis and procrastination. But it&#x27;s not the case here.

This is one of those decisions that confuses you when you&#x27;re on a mac and things just don&#x27;t work how you&#x27;d expect.

To not like GPLv3 one has be a DRM proponent. That checks out for Apple.<p>In practice though authors of GPLv3 see it as a clarification of GPLv2, i.e. they should have the same practical intent.

As I keep telling, GNU&#x2F;Linux had a lucky moment sidestepping the whole issue of AT&amp;T trying to get control back from UNIX, had this not taken place and everyone would be using classical UNIXes, with some BSD code running on them.<p>You see the same on embedded as well, all new kids on the block as embedded FOSS OSes or bare metal libraries, are either Apache or MIT licensed.

I feel compelled to comment just to note the vintage WordPress Theme being used!<p>Worth a click just to see how we used to live!

&quot;...Apple decided that while it could comply with the terms of GPLv2 license with regards to rsync 2.x, it could not comply with the terms of GPLv3 license with regards to rsync 3.x.&quot;<p>This is due to the software patent terms that appeared in GPLv3.<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=21645618">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=21645618</a><p><a href="https:&#x2F;&#x2F;lobste.rs&#x2F;s&#x2F;8lbh1k" rel="nofollow">https:&#x2F;&#x2F;lobste.rs&#x2F;s&#x2F;8lbh1k</a><p><a href="https:&#x2F;&#x2F;archive.ph&#x2F;AeMTz" rel="nofollow">https:&#x2F;&#x2F;archive.ph&#x2F;AeMTz</a>

Am I the only one that has had some hard to pin down problems with rsync? I&#x27;m excited about this because I&#x27;d love to have an alternative implementation. In particular, &quot;rsync --compress&quot; over SSH seems to have a rare and hard to track down issue. I&#x27;ve used rsync for decades doing hundreds of nightly system backups, and maybe once a month one of them goes out to lunch (IIRC it just hangs). The rarity of it makes it hard to isolate or come up with a reproduction. Removing the &quot;--compress&quot; resolves it. Anyone else ever come across something like that?

Strange, I&#x27;ve always thought about GPLv3 as of an upgrade (i.e. better license) to GPLv2

This hardly matters as any power user will keep their own toolset maintained from brew I guess.<p>And relying on open source CLI tools in Macintosh for helping end users is not a good idea.<p>What this signals to me most of all is &quot;oh we can&#x27;t steal from GNU anymore so we&#x27;ll steal from openbsd&quot;.<p>Because even if it is a neglible part of the appeal of Macintosh computers, they still make an effort to ship these tools with their OS and they make a lot of money doing it.

Apple&#x27;s shift away from GPL has been showing for years, so this doesn&#x27;t surprise me. But dropping rsync 2.6.9 this late without full 3.x support feels like a step backwards, especially for anyone relying on metadata preservation. I get the licensing angle, but the lack of feature parity in opensync might catch a lot of people off guard.

Good - MacOS had an old and crappy version of rsync

I ran into this issue too since I implemented an --rsh wrapper script (based on <a href="https:&#x2F;&#x2F;github.com&#x2F;kubernetes&#x2F;kubernetes&#x2F;issues&#x2F;13776">https:&#x2F;&#x2F;github.com&#x2F;kubernetes&#x2F;kubernetes&#x2F;issues&#x2F;13776</a>) and the options passed to ssh are different (samba rsync passes in -l user host command, openrsync passes in user@host command).<p>&gt; openrsync accepts only a subset of rsync’s command line arguments.<p>I have not upgraded to MacOS Sequoia yet so I cannot verify but from the source (<a href="https:&#x2F;&#x2F;github.com&#x2F;apple-oss-distributions&#x2F;rsync&#x2F;blob&#x2F;rsync-91.60.3&#x2F;rsync.wrapper.c">https:&#x2F;&#x2F;github.com&#x2F;apple-oss-distributions&#x2F;rsync&#x2F;blob&#x2F;rsync-...</a>) it appears that there is a wrapper and they ship both samba rsync and openrsync and fall back to samba if you use an unsupported option?

How is it these days for other developers actually banking on certain tools to be present &#x2F; working. What I mean is that apple isn’t shipping rsync etc to help us developers but because the system needs it. It was already mentioned that this issue also exists for the other cli tools because they ship the BSD not the Gnu versions. Which brings me to POSIX which was introduced back then to tackle just that. Make sure that a set of tools has defined options &#x2F; behavior etc. it seems to me that we lapsed here because more and more systems ship very custom setups which are not compatible. Or see the Linux and the issue with binary compatibility and the state of flatpack vs Snap vs others. I fear it becomes harder and harder to create cross platform solutions by using the system provided packages. Writing a shell script is already a challenge.

I stopped treating Mac OS as a Unix and I started sleeping at night. It’s a great platform for running a Unix in a VM.

It looks like there are quite a few glitches in Apple&#x27;s openrsync. Deletion of two files resulted in four deletion entries.<p><pre><code> rsync -av --delete --dry-run a&#x2F; b&#x2F; Transfer starting: 2 files folder1&#x2F;file2: deleting folder1&#x2F;file1: deleting .&#x2F;folder1&#x2F;file2: deleting .&#x2F;folder1&#x2F;file1: deleting </code></pre> Also, if a directory has extended attributes, `rsync -av --extended-attributes` resulted in `._` folders in the destination directory.<p>It is not an acceptable replacement for the original rsync.

Thanks for bringing attention to it. Did `brew install rsync` immediately, problem solved, fuck Apple.

Huh, interesting. I hadn&#x27;t noticed when I upgraded, but I don&#x27;t use many of the features of `rsync` to begin with. I ended up installing the real `rsync` shortly thereafter.

Does openrsync work?<p>The problem with Apple&#x27;s ancient userspace is so many of the utilities are outdated and don&#x27;t support things like files bigger than 4GB. So switching to a tool updated in the last 19 years may be an improvement. But then rsync is such a standard, is openrsync 100% compatible?<p>The need to install and maintain Homebrew was a big part of why I switched from MacOS to Windows. WSL is a very good Unix environment, being just Ubuntu or Debian.

In general, Apple has had such a positive influence on both hardware and software that I welcome their particular approach. It may not be ideal from a pure Linux perspective, but it does open the door to a variety of approaches. And truly, that is the key - there should always be multiple licenses and approaches to the work of open source.<p>So, thank you, Apple.<p>But please open source System 7.

It broke the `--log-files` CLI switch that I used to backup my local files to shared storage :-(. Had to reinstall rsync via homebrew to get it back

Beware if you want to copy files with extended attributes, ACLs or resource forks to another Mac.<p>The short syntax used to be &quot;rsync -Eva&quot;<p>Since openrsync this is now &quot;rsync -va ---extended-attributes&quot;<p>I will need to go through a bunch of scripts now I&#x27;m afraid. Bye bye Eva.

Can someone do an analysis of the packages of macOS Sequoia that are still GPL licenced?

I wonder if this is why Time Machine started taking up all my CPU resources after I upgraded? I had to shut off automated Time Machine backups because it literally makes my M1 MBA unusable for few minutes every hour.

Is good that we see BSD software thriving.

And yet I don&#x27;t see Apple on the list of contributors of the OpenBSD Foundation (<a href="https:&#x2F;&#x2F;www.openbsdfoundation.org&#x2F;contributors.html" rel="nofollow">https:&#x2F;&#x2F;www.openbsdfoundation.org&#x2F;contributors.html</a>), shame.

Should I be embarassed for my bash alias?<p>alias rsy=&quot;rsync -avP&quot;<p>I do this with many unix utils that have insensible (imo) defaults

Yeah, more MacOS utilities that accept slightly different arguments.

Why can’t the developers just release a licensable corporate version and Apple just agree to pay the corporate license fee?

Am I the only one finding that &quot;openrsync&quot; sounds like &quot;rsync&quot; is not open source? I find it a bit confusing because rsync is GPL.<p>Just like I would find it weird for a project to be called openlinux or librelinux...<p>Still it&#x27;s great to have multiple implementations, of course!

You don&#x27;t have to embed a github gist to show 5 lines of console output that are not even highlighted. You can use the HTML &lt;code&gt; tag.<p><a href="https:&#x2F;&#x2F;developer.mozilla.org&#x2F;en-US&#x2F;docs&#x2F;Web&#x2F;HTML" rel="nofollow">https:&#x2F;&#x2F;developer.mozilla.org&#x2F;en-US&#x2F;docs&#x2F;Web&#x2F;HTML</a>

It has differences in the command lines &#x2F; behavior. We discovered this last week.

Lol. Apple&#x27;s war on GPL 3 again. Same reason they replaced bash with zsh.<p>I&#x27;m glad I&#x27;m no longer using their stuff.<p>And yes I would need a complete implementation obviously.

I already replace the bundled rsync on mac with the proper one, as the bundled one is ancient and is missing some features I like. Same for grep, awk, sed, find<p>Mac OS is getting a bit worse every release, clearly trending towards an iOS world where we have to ask apple for permission to run anything, even in a sandbox

After decades using rsync for my local backups, I recently switched to ChronoSync Express. It&#x27;s simple to use, with a sensible GUI and well-laid-out customization options.<p>And btw, it&#x27;s included on Setapp subscriptions.