Question

I’m passingly familiar with the dd command, but I’ve rarely had the need to use it myself. Today I need to, but I’m encountering behavior that seems really weird.

I want to create a 100M text file, each line of which contains the single word “testing.” This was my first try:

~$ perl -e 'print "testing\n" while 1' | dd of=X bs=1M count=100
0+100 records in
0+100 records out
561152 bytes (561 kB) copied, 0.00416429 s, 135 MB/s

Hmm, that’s odd. What about other combinations?

~$ perl -e 'print "testing\n" while 1' | dd of=X bs=100K count=1K
0+1024 records in
0+1024 records out
4268032 bytes (4.3 MB) copied, 0.0353145 s, 121 MB/s

~$ perl -e 'print "testing\n" while 1' | dd of=X bs=10K count=10K
86+10154 records in
86+10154 records out
42524672 bytes (43 MB) copied, 0.35403 s, 120 MB/s

~$ perl -e 'print "testing\n" while 1' | dd of=X bs=1K count=100K
102400+0 records in
102400+0 records out
104857600 bytes (105 MB) copied, 0.879549 s, 119 MB/s

So of these four apparently-equivalent commands, all produce files of different sizes, only one of which is the one I would expect. Why is that?

EDIT: By the by, I’m a little embarrassed I didn’t think of “yes testing” instead of that longer Perl command.

Answer 1

I’m not yet sure why, but using this method will not fill up an entire block before saving it. Try:

perl -e 'print "testing\n" while 1' | dd of=output.txt bs=10K count=10K iflag=fullblock
10240+0 records in
10240+0 records out
104857600 bytes (105 MB) copied, 2.79572 s, 37.5 MB/s

The iflag=fullblock seems to force dd to accumulate input until the block is full, although I’m not sure why this is not the default, or what it actually does by default.