Question

I’m in the design phase of a website and I have a solution for a feature but I don’t know if it will be the good one when the site, hopefully, grows. I want the users to be able to perform searches for other users and the results they find must be ordered: first the “spotlighted” users, then all the rest. The result must be ordered randomly, respecting the previously mentioned order, and with pagination.

One of the solutions I have in mind is to store the query results in a session variable in the server side. For performance, when the user leaves the search this variable is destroyed.

What will happen when the site has thousands of users and every day thousands of searches are performed? My solution will be viable or the server will be overloaded?

I have more solutions in mind like an intermediate table where n times by day users are dumped in the mentioned order. This way there is no need to create a big array in the user’s session and pagination is done via multiple queries against the database.

Although I appreciate any suggestions I’m specially interested into hear opinions from developers seasoned in transited sites.

(The technology employed is LAMP, with InnoDb tables)

Answer 1

Premature optimization is bad. But you should be planning ahead. You dont need to implement it. But prepare yourself.

If there are thousands of users searching this query everyday then caching the query result in session is not a good idea. Cause same result can be cached for some users while other needs to execute it. For such case I’d recommend you save the search result in user independent data structure (File, memory etc).

1. For each search query save the result, creation date, last access date in your disk or in memory.
2. If any user searches the same query show the result from cache
3. Run a cron that invalidates the cache after sometime.

This way frequent searches will most time promptly available. Also it reduces the load on your database.