Question

I’ve put together a PHP function as follows:

function keyword_hash($keyword) {
  return base_convert(substr(md5($keyword), -16), 16, 10);
}

The aim of this function is to generate a numeric hash value that I can store in a database, and use that for a look up (rather than trying to index a keyword column).

The equivalent of this function in MySQL is as follows:

SELECT CONV(RIGHT(MD5('some keyword'), 16), 16, 10);

I’ve verified that the MD5 string is the same, and the substr() matches the value I get back from RIGHT() in the MySQL query. However, when I run CONV(), I’m getting a different value to what’s generated from base_convert().

For example, using keyword_hash("some keyword") generates a value of 10923672322315740844. However, using SELECT CONV(RIGHT(MD5('some keyword'), 16), 16, 10) generates 10923672322315740475, which shows the last three numbers as being different.

What am I missing here? Shouldn’t they produce the same value?

Answer 1

I took a look at the PHP Manual page for base_convert(). There is the following warning:

base_convert() may lose precision on large numbers due to properties related
to the internal "double" or "float" type used. Please see the
Floating point numbers section in the manual for more
specific information and limitations.

Later in the comments someone already found a solution for this problem (thanks @CraigSefton):

function str_baseconvert($str, $frombase=10, $tobase=36) { 
    $str = trim($str); 
    if (intval($frombase) != 10) { 
        $len = strlen($str); 
        $q = 0; 
        for ($i=0; $i<$len; $i++) { 
            $r = base_convert($str[$i], $frombase, 10); 
            $q = bcadd(bcmul($q, $frombase), $r); 
        } 
    } 
    else $q = $str; 
  
    if (intval($tobase) != 10) { 
        $s = ''; 
        while (bccomp($q, '0', 0) > 0) { 
            $r = intval(bcmod($q, $tobase)); 
            $s = base_convert($r, 10, $tobase) . $s; 
            $q = bcdiv($q, $tobase, 0); 
        } 
    } 
    else $s = $q; 
  
    return $s; 
}

This function uses the bc math library which supports arbitrary precision mathematics because it uses strings to store the numbers instead of integers/floats etc.