Question

The Adler-32 checksum algorithm does sums modulo 65521. I know that 65521 is the largest prime number that fits in 16 bits, but why is it important to use a prime number in this algorithm?

(I’m sure the answer will seem obvious once someone tells me, but the number-theory parts of my brain just aren’t working. Even without expertise in checksum algorithms, a smart person who reads http://en.wikipedia.org/wiki/Fletcher%27s_checksum can probably explain it to me.)

Answer 1

Why was mod prime used for Adler32?

From Adler’s own website http://zlib.net/zlib_tech.html

However, Adler-32
has been constructed to minimize the
ways to make small changes in the data
that result in the same check value,
through the use of sums significantly
larger than the bytes and by using a
prime (65521) for the modulus. It is
in this area that some analysis is
deserved, but it has not yet been
done.

The main reason for Adler-32 is, of
course, speed in software
implementations.

An alternative to Adler-32 is Fletcher-32, which replaces the modulo of 65521 with 65535. This paper shows that Fletcher-32 is superior for channels with low-rate random bit errors.

It was used because primes tend to have better mixing properties. Exactly how good it is remains to be discussed.

Other Explanations

Someone else in this thread makes a somewhat convincing argument that modulus a prime is better for detecting bit-swapping. However, this is most likely not the case because bit-swapping is extremely rare. The two most prevalent errors are:

1. Random bit-flips (1 <-> 0) common anywhere.
2. Bit shifting (1 2 3 4 5 -> 2 3 4 5 or 1 1 2 3 4 5) common in networking

Most of the bit-swapping out there is caused by random bit-flips that happened to look like a bit swap.

Error correction codes are in fact, designed to withstand n-bits of deviation. From Adler’s website:

A properly constructed CRC-n has the
nice property that less than n bits in
error is always detectable. This is
not always true for Adler-32–it can
detect all one- or two-byte errors but
can miss some three-byte errors.

Effectiveness of using a prime modulus

I did a long writeup on essentially the same question. Why modulo a prime number?

http://www.codexon.com/posts/hash-functions-the-modulo-prime-myth

The short answer

We know much less about prime numbers than composite ones. Therefore people like Knuth started using them.

While it might be true that primes have less relationship to much of the data we hash, increasing the table/modulo size also decreases the probability of a collision (sometimes more than any benefit gained from rounding down to the nearest prime).

Here is a graph of collisions per bucket with 10 million cryptographically random integers comparing mod 65521 vs 65535.