Question

all.

I want to use gunzip to calculate a string’s crc32 code.

String: teststring

This command works well:
echo -n teststring | gzip -f > /tmp/filename && gunzip -lv /tmp/filename && rm -f /tmp/filename

But this command can’t work:
echo -n teststring | gzip -f | gunzip -lv

I searched all the gunzip manual, and find this “The –list option reports sizes as -1 and crc as ffffffff if the compressed file is on a non seekable media.”, I want to why the standard output is a non seekable media? How can make this command works well?

Update:

This does’t work:

cat /tmp/filename | gunzip -lv


This works well:

gunzip -lv < /tmp/filename


Can you tell me the difference between these two commands? Thank you!

Answer 1

When you pipe, the input to the receiving executable is simply not seekable. Hence the problem you are experiencing.

Instead you could extract the second-to-last four bytes, which is the crc:

% echo -n teststring | gzip -1 | tail -c 8 | head -c 4 | hexdump -e '1/4 "%08x" "\n"'
1f58f83e

or you could use pigz (a parallel version of gzip that happens to not try to seek for a listing):

% echo -n teststring | gzip -1 | pigz -lv
method    check    timestamp    compressed   original reduced  name
gzip 8  1f58f83e  Jul 19 01:35          12         10  -20.0%  <stdin>

In general, this is a rather inefficient way to calculate the crc, since gzip is doing a lot more work to compress than it is to calculate the crc. (I put in the -1 option above to minimize the compression work, but it’s still a lot more.) You should simply write a short program to calculate the crc. Your linux machine almost certainly has the zlib library available. Try to include zlib.h and use -lz to link. You can read zlib.h to see how to use the crc32() function.

The cksum command suggested in a comment can be used to calculate a different crc. If all you want is any crc, then you can use that. If you want the crc used by gzip, zip, png, etc., then you need to use the crc32() routine from zlib, or equivalent.