Question

I have an active record Song model with a songhash field (string(255)) that contains a Sha2 hash. When I try to find a song via the following code nothing gets returned:

song = Song.all.first
song2 = Song.where(songhash: song.songhash).first
# song is a valid object with a songhash set, but song2 is nil!

If I do the same thing however with a “like” query it works:

song = Song.all.first
song2 = Song.where("songhash like ?", song.songhash).first
# song2 is a valid object now
song2.songhash == song.songhash
# the equation is true

I fear it has something to do with string encodings but I have no idea why this string could possibly have encoding issues: 61a9761b9ebd543b72c5ccf2ab6db198b067f7cf7f8412ee6e9c14b19611bc80

I’m using rails 3.1 with sqlite db.

Any ideas what’s going on?

Answer 1

Thanks to help of @gazler and @basgys I was able to track down the problem:

It is in fact an encoding problem caused by the Digest::Sha2#hexdigest function. It returns a string that is encoded as ASCII-8BIT. When storing this in the database it seems to be converted automatically to a UTF-8 string (I check that by running a select hex(songhash) from songs query). However when using the string in the query, it does not seem to do that conversion.

Internally ruby seems to handle the different encoding conversions automatically. That is why "abc"=="abc" although they may have different encoding.

I’m sure that this is not expected behavior, however I don’t know if it is a bug – and if it is a bug whether it is somewhere within ActiveRecord, the SQLite Driver, or SQLite itself.

My solution is now to append a .encode("UTF-8") to the result of the digest function.