SCP – Familiar, Simple, Insecure, and Slow

"SCP – Familiar, Simple, Insecure, and Slow" - slightly sensationalized title. All you need to know - if you scp with an evil machine, bad things can happen. That's all.

I&#x27;ve gotten into a habit of setting `alias scp=rsync` (with compression, partial, progress, etc args usually) on every system I use. In the once-in-a-lifetime cases where rsync is <i>not</i> pre-installed, I just call &#x2F;usr&#x2F;bin&#x2F;scp directly (or wherever it happens to be. `type -a scp` will tell you.)<p>The amount of time I&#x27;ve saved when I&#x27;ve <i>forgotten</i> that I&#x27;ve done this and it just runs faster is well into the tens of hours. The amount of time I&#x27;ve lost when I forgot and rsync doesn&#x27;t exist remotely is about 10 seconds total - enough time to read the error message -&gt; `type -a scp` -&gt; &quot;oh yeah&quot; -&gt; use &#x2F;usr&#x2F;bin&#x2F;scp instead.<p>Highly highly highly recommended.

It turns out that rsync is slower than scp, unless the files are already present. It adds a lot of overhead that’s not needed most of the time.<p>I often need to transfer very large binary files. When stuck with standard Linux tools, the fastest process is first transfer via scp. If the scp gets interrupted, resume it using rsync with the —-append-verify option.

We all use rsync but rsync itself has massive risks and is equally prone to stupid buffering consequences. (I worked on a write up of an exploit to hose an rsync server in rpki and the lack of channel protect in rsync is a problem. Rsync over SSH is less risky but not risk free to either side. What is sent and what is asked for do not have to align well) rsync over SSH needs tuning to be buffer efficient.<p>Tar may well be better if you know you need all the files and don&#x27;t need the block checks per file to detect change and minimise data sent.<p>Rsync can hose the file buffer cache and slow a server down. If you want speed mbuffer is good but lacks channel security too (ZFS send over mbuffer is awesomely fast)

Is there a tool that mimics the interface of SCP but uses SFTP under the hood? Then I could alias `scp` to that and forget about the SCP protocol forever. (rsync is not good enough: it requires rsync to be installed on the target server and seems to be slower than SCP in some cases.)<p>The proprietary ssh.com implementation of SSH apparently has one called &quot;scp2&quot;, but I don&#x27;t see any open source version.<p>Well, except for this, but it&#x27;s tagged &quot;[MAINTAINER WANTED]&quot; and written in JavaScript:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;spmjs&#x2F;node-scp2" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;spmjs&#x2F;node-scp2</a>

I feel like this article tries a little too hard to make the SCP protocol sound weird. It’s just using SSH as a transport layer, which is not really unusual at all. Perhaps there are some security issues but IIRC Git for example does basically the same exact thing to use SSH as a transport layer, calling git-receive-pack on the remote. I’d be surprised if Rsync wasn’t the same.

I always assumed that sftp was the hacky command and scp was the preferred one.<p>Guess I maybe learned that wrong.

Compared to other file transfer protocols (I’m looking at you SMB), SCP is pretty fast in my experience.

For a long time I have just piped tar over ssh. For lots of small files, it&#x27;s faster than an rsync to an empty directory. It can also sync anything that the tar on both sides can handle. I wrote a script for doing this for my coworkers who identify with <a href="https:&#x2F;&#x2F;xkcd.com&#x2F;1168&#x2F;" rel="nofollow">https:&#x2F;&#x2F;xkcd.com&#x2F;1168&#x2F;</a>