"This is a strange program of obscure provenance that somehow, still manages to survive in the 21st century."<p>-> links to wikipedia page with direct discription of lineage back to 5th ed research unix<p>"That weird bs=4M argument in the dd version isn’t actually doing anything special—all it’s doing is instructing the dd command to use a 4 MB buffer size while copying. But who cares? Why not just let the command figure out the right buffer size automatically?"<p>Um -<p>a) it is 'doing the special thing' of changing the <i>block</i> size (not buffer size)<p>b) Because the command probably doesn't figure out the right size automatically, much like your 'cat' example above which also doesn't<p>c) And this can mean massive performance differences between invocations<p>> Another reason to prefer the cat variant is that it lets you actually string together a normal shell pipeline. For instance, if you want progress information with cat you can combine it with the pv command<p>Umm:<p><pre><code> dd if=file bs=some-optimal-block-size | rest-of-pipeline
</code></pre>
that was hard.<p>>If you want to create a file of a certain size, you can do so using other standard programs like head. For instance, here are two ways to create a 100 MB file containing all zeroes:<p><pre><code> $ uname -sr
OpenBSD 6.0
$ head -c 10MB /dev/zero
head: unknown option -- c
usage: head [-count | -n count] [file ...]
</code></pre>
well.. guess that wasn't so 'standard' after all..
I must be using some nonstandard version...<p><pre><code> $ man head |sed -ne 47,51p
HISTORY
The head utility first appeared in 1BSD.
AUTHORS
Bill Joy, August 24, 1977.
$ sed -ne 4p /usr/src/usr.bin/head/head.c
* Copyright (c) 1980, 1987 Regents of the University of California.
</code></pre>
Hmm..<p>> So if you find yourself doing that a lot, I won’t blame you for reaching for dd. But otherwise, try to stick to more standard Unix tools.<p>Like 'pv'?<p>edit: added formatting, sector size note, head manpage/head.c stuffs.. apologies.
There's one good (?) reason to use dd with devices: it specifies target in the same command. For devices, writing to them usually requires root privileges, so it's easy to:<p><pre><code> sudo dd .... of=/dev/...
</code></pre>
But there's no trivial cat equivalent:<p><pre><code> sudo cat ... > target
</code></pre>
Will open target as your current user anyway. You can play around with tee and redirection of course. But that's getting more complicated than the original.
One thing I'll often use dd for is recovering data from a failing drive. Can head ignore read errors? dd can.<p>As far as I'm concerned, dd is lower-level than most of the other utilities and provides more control over what's happening.<p>The author does have a point that the syntax is strange though.
This article is full of Useless Uses of Cat[1] that could just use redirection operators. For instance,<p><pre><code> cat image.iso | pv >/dev/sdb
</code></pre>
could be rewritten as<p><pre><code> pv < image.iso > /dev/sdb
</code></pre>
A related mistake is the Useless Use of Echo, since any command of the form<p><pre><code> echo "foo" | bar
</code></pre>
can be written using here strings as<p><pre><code> bar <<< "foo"
</code></pre>
or even<p><pre><code> bar <<WORD
foo
WORD
</code></pre>
[1] <a href="http://porkmail.org/era/unix/award.html" rel="nofollow">http://porkmail.org/era/unix/award.html</a>
For those of you that are blissfully unaware of what the JCL DD command looks like, here's a example (with only the DD section of the JCL shown):<p><pre><code> //SYSPRINT DD SYSOUT=*
//SYSLIN DD DSN=&&OBJAPBND,
// DISP=(NEW,PASS),SPACE=(TRK,(3,3)),
// DCB=(RECFM=FB,LRECL=80,BLKSIZE=3200),
// UNIT=&SAMPUNIT
//SYSLIB DD DSN=SYS1.MACLIB,DISP=SHR
//SYSIN DD DSN=&SAMPLIB(IEWAPBND),DISP=SHR</code></pre>
<i>But who cares? Why not just let the command figure out the right buffer size automatically?</i><p>Because it can be a lot slower. dd is low level, hence powerful and dangerous.<p>And, if we are going down that rabbit hole, you don't need cat[1]<p>“The purpose of cat is to concatenate (or "catenate") files. If it's only one file, concatenating it with nothing at all is a waste of time, and costs you a process.”<p>[1]<a href="http://porkmail.org/era/unix/award.html#cat" rel="nofollow">http://porkmail.org/era/unix/award.html#cat</a>
The Ignorance Of Err Ignorant People<p>dd is a tool. dd can do a lot more then cat. dd can count, seek, skip (seek/drop input), and do basic-ish data conversion. dd <i>is</i> standard, even more standard then cat (the GNU breed). I even used it to flip a byte in a binary, a couple of times.<p>New-ish gnu dd even adds a nice progress display option (standard is sending it sigusr1, since dd is made to be scripted where only the exit code matters).<p>> Actually, using dd is almost never necessary, and due to its highly nonstandard syntax is usually just an easy way to mess things up.<p>Personally I never messed it up, nor was confused about it. This sentence also sets the tone of the whole article, a rather subjective tone that is.<p>edit: Some dd usage examples: <a href="http://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/" rel="nofollow">http://www.linuxquestions.org/questions/linux-newbie-8/learn...</a>
Don't cat a file and pipe it into pv. Use "pv file" as a replacement for "cat file" and it will show you the progress as a percentage. When it's in the middle of a pipeline, it doesn't know the total size (unless you tell it with -s), so it can only show the throughput.
A counterpoint: dd survives not because it's good or makes sense, but explicitly because it doesn't.<p>You wanna format a usb key? Google this, copy/paste these dd instructions, it works, move on with your life.<p>You wanna format a usb key using something related to cat you once saw and didn't fully understand? Have fun.<p>Both approaches have their weak points, but in any OS the answer to "How do I format a usb key" should not start with "Oh boy, let's have a Socratic dialog over 10 years on how to do that."
What about the `seek` argument which skips over some blocks at the beginning but still allocates them (unix "holes")?<p>Also note that there are still unix systems out there which do not support byte-level granularity of access to block devices. On those devices you must actually use a buffer of exactly the size of the blocks on the device. Heck, linux was like this until at least v2.
I think dd is primarily so popular because it is used in mostly dangerous operations. Sure, using cat makes logicial sense, but if we are talking about writing directly to disk devices here I'll trust the command I read from the manual and not explore commands I <i>think</i> would work.<p>dd's "highly nonstandard syntax" comes from the JCL programming language, but it's really just another tool to read and write files. At the end of the day it's not more complex or incompatible than other unix tools. For example, you can also use tools like `pv` with dd no problem to get progress statements.
Cult of pv. It looks to have more command-line complexity than dd. <a href="https://linux.die.net/man/1/pv" rel="nofollow">https://linux.die.net/man/1/pv</a>
This is a great example of why downvoting submissions should be a thing. Or at least showing the up/down tuple. I would say every upvote represents someone misled and likely to further propagate this nonsense.
The author doesn't even give correct invocations of dd (on BSD, at least, for their last example with head).<p>I certainly agree the syntax of the arguments is strange, due to its age, but I don't agree that learning it is difficult or a waste of time.<p>All I've learned is that the author doesn't like dd well enough to learn it.
OP, your alternatives to DD are more complicated, not less complicated. I shouldn't need to pipeline two commands together just to cut off the first 100MB of a file.
Dude's missing an important point:<p>If you mess up the syntax on a dd invocation, a nice thing happens: nothing.<p>Use a shell command and pipes, and your command better be perfect before you hit return.
Somewhat related short story: Earlier this week my friend said that he dd'd away just over 50 bitcoins, back when they were worth ~$3 each.<p>"One of the biggest regrets of my life."
I'll point out that dd also allows you to control lots of other filesystem and OS-related things that other tools do not. See: fsync/fdatasync. I'm not aware of any shell tools that allow you to write data like that.
An even easier solution: don't make people fall into the command line to format a USB reliably.<p>The command line should be reserved for times where you need the fine grain control to do something that DD is meant to do. A GUI should implement everything else in a reliable way that doesn't break half the time or crash on unexprected input.
Ignorance on the blocksize arg.<p>Also, I only need to remember one progress command for my entire operating system: control+t. I also get a kernel wait channel from that which is phenomenally pertinent to rapidly understanding and diagnosing what the heck a command is doing or why it is stuck.<p>I hate what Linux has done to systems software culture.
Specifiying a large block size used to help a LOT with performance. From memory shell redirection used a tiny blocksize. On Solaris at least.<p>And <i>if</i> you use dd then you probably should specify a bigger block size than the default of 512 bytes.<p>But yeah, most usage is obsolete.
I think this article is full of "alternative computer science" and reminds me other article, published here as well, about the obsolescence of Unix. The only good thing is this discussion thread.
Interesting assertion. Can you show me a shell invocation without using dd that cuts off the first 16 bytes of a binary file, for example? This is a common reason I use dd.
dd precisely controls the sizes of read, write and lseek system calls. This doesn't matter on buffered block devices; there is no "reblocking" benefit.<p>Some kinds of devices are structured such that each write produces a discrete block, with a maximum size (such that any bytes in excess are discarded) and each read reads only from one block, advancing to the next one (such that any unread bytes in the current block due to the buffer being too small are discarded). This is very reminiscent of datagram sockets in the IPC/networking arena. dd was developed as an invaluable tool for "reblocking" data for these kinds of devices.<p>One point that the blog author doesn't realize (or neglects to comment upon) is that "head -c 100MB" relies on an extension, whereas "dd if=/dev/zero of=image.iso bs=4MB count=25" is ... <i>almost</i> POSIX: there is no MB suffix documented by POSIX, only "b" and "k" (lower case). The operator "x" is in POSIX: bs=4x1024x1024.<p>Here is a non-useless use of dd to request exactly one byte of input from a TTY in raw mode:<p>file:///usr/share/doc/bash-doc/examples/scripts/line-input.bash<p>Wrote that myself, back in 1996; was surprised years later to find it in the Bash distribution.
My most common use of dd is warming up AWS EBS volumes. <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html" rel="nofollow">http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initi...</a><p>Though fio is better because it can work in parallel.
dd is for handling blocked data, while cat, redirection and pipelines are completely useless for that, since they are not meant to manipulate blocks of data, but streams. They do not compare (apart from really simple cases where either will do, like copying a file into some other file); this blog posts mainly highlights that neither the author nor many tutorial writers now the difference.