Question

I have checked different implementations of CRC64. For example, this, this and this. The problem with all these is that they work with bytes. However, on a 64 bit system, I would like to work with long (8 bytes). In this way, I will need to iterate less. For example, for data of 128 bytes, using a byte, I need to iterate 128 times, while with long, I would need to iterate only 16 times.

Is there any CRC64 implementation that use long or even a word size greater than a byte? Can these schemes be modified to do so?

Answer 1

CRC calculation uses a trick to avoid having to process the data bit-by-bit: It uses a lookup table which allows it to process multiple bits at once.

Processing n bits at once requires a lookup table of size 2^n. The implementations you linked read 1 byte (8 bits) at a time, and indeed they all use a lookup table of size 256 == 2^8.

Processing 64 bits at a time would require a lookup table of size 2^64, which is not practical. This is why common implementations of CRC do the processing 1 byte at a time.

While it’s possible to process 2 bytes at a time using a 65536-entry array, this is likely to have a negative performance impact due to using more CPU cache memory.