Question

I am using rails and for search I am using Tire and elasticsearch. I have a string type field which in some records have value and in some records is nil.

I’d like to sort and show last, all the records that have null value in this field. As I see in this issue https://github.com/elasticsearch/elasticsearch/issues/896 in the current version this can’t be possible through sort and elasticsearch.

Is there a workaround with rails? I am trying to do it using two searches and using filters like the following example:

    filter :not, :missing => { :field => :video_url } if params[:video].present?
    filter :missing, { :field => :video_url } if params[:video].blank?

But it didn’t work (I can’t understand why until now, I’ll continue debugging).

Another idea is to create two methods with the specific fields. Any other solution/idea?

Update 2/2/2013

I finally did it like the following:

    if video == "yes"
      filter :not, :missing => { :field => :video_url }
    elsif video == "no"
      filter :missing, { :field => :video_url }
    end

And I am passing the video parameter by my own. I am sorting and boosting the search but additionally I want all the objects that hasn’t got video_url field, to appear at the bottom no matter how relevant they are. Indeed I don’t need to sort by this field, just to show last the nil value fields.

So to solve this I am calling two times the search and with the addition of the code above, it works like a charm.

Just for completeness, my search method is the following:

  def self.search(params, video = nil)
tire.search do
  query do
    boolean do
      must { string params[:query], default_operator: "AND" } if params[:query].present?
      must { term :active, true }
    end
  end
    sort { by :update_ad => "desc" } unless params[:query].present?
    facet "categories" do
      terms :category_id
    end
    if video == "yes"
      filter :not, :missing => { :field => :video_url }
    elsif video == "no"
      filter :missing, { :field => :video_url }
    end
end

end

If you don’t pass the video param, it won’t apply any filter. In my mapping, I have set the boost, analyzers etc.

Thank you

Answer 1

First, the Elasticsearch issue you’re linking to is still open and is only a feature suggestion.

Second, just as a note, are you really sure you want to sort as opposed to boost the score of certain records?

Third, if you indeed do want to sort on this field, the easiest way is to just index the field with some value which comes last ("ZZZ", weird Unicode chars, you get the picture). You probably don’t want to do this by default, so it’s a good idea to use the multi_field feature. Of course, you have to reindex your corpus to pick up the new settings.

Lastly, it is possible to sort by a script (see documentation), but it has the usual and obvious performance impact.