Question

I am trying to find out if there is any API in C for calculating a 64 bit hash.
I found out that some people use top 64 bits of MD5/SHA1 etc. Is it a good approach?

Answer 1

You could try SipHash in its form as a MAC (which requires key management, though). It is particularly well-suited for short input messages and aims at cryptographic strength. A C implementation is also available.

But if you really care about someone actively messing around with your files, you shouldn’t restrict yourself to 64 bits of security. 64 bits can be broken even by brute force today, given enough time and resources. You should use SHA-256 or stronger for that. Or let me state it the other way round, blacklisting broken options: don’t use MD5 (or MD-anything for that matter). Use SHA-1 only if you can’t use SHA-256 for some reason.

Using a hash also has the advantage that you don’t need to manage any keys (opposed to using a MAC). You should just keep the hashes you compute in a different place than the files you are about to monitor – otherwise somebody tampering with your files can easily tamper with the checksum, too.

Regarding whether truncating hashes is good or bad

In theory, I can’t see why it should be wrong to truncate a let’s say 160 bit hash value down to 64 bit, regardless of whether you take the most significant bits or the least significant bits or pick them using any arbitrary pattern. The only reason why this isn’t done more often that I can think of is efficiency – why bring the big guns if there are more efficient algorithms for handling the smaller problems.

In what follows, I assume a cryptographically secure hash for this purpose, general-purpose hashes are quite a different topic – they might expose attack surfaces when truncated for all I know.

But, for a cryptographically secure hash, unless the algorithm is broken, we can assume that its output is indistinguishable from that of a uniformly distributed random variable.

If we truncate this value now, we don’t offer any further insight into the inner workings of the algorithm. Still, we do weaken the security by the simple fact that brute-forcing (be it collisions or finding pre-images) now takes less time by laws of probability.

For example, finding a collision for a 64 bit hash takes roughly 2^32 attempts on average – says the Birthday Paradoxon. If you truncate your output down to the least significant 32 bits of the original 64 bit hash, then you will find collisions in time roughly 2^16, because you simply ignore the most significant 32 bits and the de-facto uniform distribution does the rest – it’s like you started searching for collisions with a 32 bit value in the first place.