I use MD5 hash for identifying files with unknown origin. No attacker here, so I don’t care that MD5 has been broken and one can intendedly generate collisions.
My problem is I need to provide logging so that different problems are diagnosed easier. If I log every hash as a hex string that’s too long, inconvenient and looks ugly, so I’d like to shorten the hash string.
Now I know that just taking a small part of a GUID is a very bad idea – GUIDs are designed to be unique, but part of them are not.
Is the same true for MD5 – can I take say first 4 bytes of MD5 and assume that I only get collision probability higher due to the reduced number of bytes compared to the original hash?
The short answer is yes, you can use the first 4 bytes as an id. Beware of the birthday paradox though:
http://en.wikipedia.org/wiki/Birthday_paradox
The risk of a collision rapidly increases as you add more files. With 50.000 there’s roughly 25% chance that you’ll get an id collision.
EDIT: Ok, just read the link to your other question and with 100.000 files the chance of collision is roughly 70%.