I need to build a system (in PHP) that does massive queries on MSSQL to retrieve the result as follows:
I have 10 independent queries that I need to run when a user visits a page (some are massive). What is the most scalable solution to running 3 queries in parallel (7 in queue) and when a query is finished, run another from the queue?
I need the page to load instantly, and as the query finishes, show the user the response of that query (the order is not so important).
I took into consideration: AJAX, beanstalkd, NodeJS, libevent and Gearman or implement it as a web service in java (using threads and parallel processing).
PS : I also found AJAX PUSH ENGINE (APE - http://www.ape-project.org/) and xRML (http://www.xrtml.org/) but never used it
NOTE this answer is for the first revision of the question that did not ask for MSSQL in particular
MySQL supports asynchronous queries via MySQLi when using the mysqlnd driver.
Passing
MYSQLI_ASYNC(available with mysqlnd) as$resultmodeargument tomysqli_query()performs the query asynchronously.mysqli_poll()andmysqli_reap_async_query()is then used to get results from such queries.Also see http://blog.ulf-wendel.de/?p=170 for an in-depth discussing of parallel, background and asynchronous fetch ideas for mysqlnd.