Question

Something that I have been thinking about lately is how do sites such as imdb or github have such fast search recommendations as I type in the search bar. Most jQuery plugins I found that do this dump the data set on to the page and then play with it there. But this would not work with large sets of database as it would make page weight colossal. My first idea was possibly a synchronous ajax call, but I was told that this could cause freezing until execution had completed.

The question: How to pull like records from a large data set while typing into a form and then display the results.

Answer 1

I recently built something like this for a FAQ, the 2 biggest problems you face are the follow:

• Database read time, the amount of time it takes to run your search query on your DB
• Your call time, if someone type 3 characters in a second then you are doing 3 ajax calls in a second.

If it is a very basic search the first issue isn’t such a big issue, you can easily query a smallish table in milliseconds, but when search mass text it gets slow. The best way to get past the is to get a search engine like Sphinx or Solr. They index data from your database for quick and relevant look ups.

The second is solved with a perception, when there is a break of less than a second (this depends on the computer literacy of your audience) between the users key strokes you DO NOT do the look up. that means that only when they slightly pause a lookup will take place. This means fewer queries and less strain on your system. Example below:

var typingTimer;
var doneTypingInterval = 1000;

$('.huge-input').stop().keyup(function(e){
    if ((e.which <= 90 && e.which >= 48) || e.which == 8){
        var myHugeInput = $(this);

        clearTimeout(typingTimer);
        typingTimer = setTimeout(
        function(){
                //Do the look up and set data
            },
            doneTypingInterval
        );
    }
});

So here we check on an input box for a valid key enter, we clear the timer and then set a time out for 1000 milliseconds (1 second), when the time runs out it does the search and sets the data, but if there is another valid key stroke in that second it clears the timer and resets it.

Hope this helps.