I have an ajax function which call a servlet to get list of products from various webservices, the number of products can go up to 100,000. I need to show this list in a html table.
I am trying to provide users an interface to filter this list based on several criteria. Currently I am using a simple jQuery plugin to achieve this, but I found it to hog memory and time.
The Javascript that I use basically uses regex to search and filter rows matching the filtering criteria.
I was thinking of an alternate solution wherein I filter the JSON array returned by my servlet and bind the html table to it. Is there a way to achieve this, if there is, then is it more efficient than the regex approach.
Going through up to 100,000 items and checking if they meet your criteria is going to take a while, especially if the criteria might be complex (must be CONDO with 2 OR 3 bedrooms NOT in zip code 12345 and FIREPLACE but not JACUZZI).
Perhaps your servlet could cache the data for the 100,000 items and it could do the filtering, based on criteria posted by the user’s browser. It could return, say, “items 1-50 of 12,456 selected from 100,000” and let the user page forward to the next 50 or so, and even select how many items to get back (25, 50, all).
If they select “all” before narrowing down the number very far, then a halfway observant user will expect it to take a while to load.
In other words, don’t even TRY to manage the 100,000 items in the browser, let the server do it.