Question

Situation:
I have user model. attribute “meta_data” in db represents “text” type field.
In model it seriazized by custom class. ( serialize :meta_data, CustomJsonSerializer.new )

It means, when I have an instance of user, I can work with meta_data like with Hash.

User.first.meta_data['username']

Problem:

I need to write a search function, which will search users by given string. I can do it by manual building search query in rails ex. User.where("email LIKE '%#{string}%'")...
But what about meta_data ? Should I search in this field by LIKE statement too? If I will do so, it will decrease relevance of found record.

For example:

I have 2 users. One of them has username “patrick”, another one is “sergio”

meta data in db will look like this:

1) {username: patrick}

2) {username: sergio}

I want to find sergio , I enter a search string “ser” => but I have 2 results, instead of one. This meta_data string “{uSERname: Patrick}” also has “ser”, so it makes this record irrelevant.

Do you have any idea how to solve it?

Answer 1

That’s really the problem with serialized data. In theory, the serialization could be an algorithm that is very unsearchable. It could do a Hoffman encoding, or other compression, and store the serialization in binary. You are relying on the assumption that the serialization uses JSON and your string will still be findable as a sub-string in the serialization.

Then the problem you are having is another issue. Other data in the serialization can mess up your results.

In general, if you serialize data, you are making a choice to not be searchable.

So a solution would be to add an additional field that you populate in a way that you control. Have a values field and store a pipe (|) delimited value that you can search. So if the data is {firstname: “Patrick”, lastname: “Stern”}, your meta_values field might be “Patrick|Stern”.

Also, don’t use the where method with a string with #{} expansion of input values. The makes it vulnerable to SQL attacks. Instead use:

where("meta_values is like :pattern", pattern: "%#{string}%")

I know that may not look very different, but ActiveRecord will go through a sanitizing this way. If someone has a semi-colon in string, then ActiveRecord will escape the semi-colon in the search condition.