Question

My ASP Classic application are fetching over 10.000 rows via a triple inner join.
After this, it is going through each line, calculating some stuff and putting data in a multidimensional array with 17 coloumns.

All this are being fetched through a jQuery AJAX when pushing the search button.

Logically this takes a while, 5+ minutes actually.

It all works fine. BUT… While doing this, the whole system freezes, not only for the user doing the calculations, but also for all the other users using the system in other ways.
How can I optimize this?!

A small relevant code snippet:

dim userArray()

sql = "select first.*, second.fullname, second.info, third.inputs from first inner join second on first.userid = second.id inner join third on first.info = third.id where convert(varchar, first.period, 112) between '20020115' and '20120115' order by second.fullname, first.userid"
set rs = conn.execute(sql)

if rs.eof then
  response.write("Nothing were found in the period")
else
  counter = 0
  redim userArray(17, 1000)
  do until rs.eof
    response.flush
    counter = counter + 1

  ' A LOT of calculations and putting in array...

    rs.movenext
  loop

  for i = 1 to counter
    ' Doing all the response.writes to the user...
  next
end if

Answer 1

Lets analyse this whilst also bearing mind the SQL has an ORDER BY Clause:-

do until rs.eof 
  response.flush 
  counter = counter + 1 

  ' A LOT of calculations and putting in array... 

  rs.movenext 
loop

Note the Response.Flush, first thing I would do is get rid of that. You will probably need to increase the ASP Response Buffering Limit (in IIS manager). Flush is sending the generated content so far to the client, it waits for the client to acknowledge receipt of all the packets sent before it completes. That is where I’m gonna guess 90% of the 5+ minutes is being spent.

Now “A LOT calculations”. VBScript is not know for its peformance. This code may well take some time. In some cases some calculations can be done much better by SQL than in script so that is one option. Another would be to build some COM compiled component to do complex work (although some accounting needs to made for marshalling which can wipe out benefits). However it may be unavoidable that you need to do these calcs in VBScript.

Now rs.movenext. This loop means you hold the connection and rowset open for pretty much all the time the processing is required. That is while the servers is sending bytes over the network to the client and while VBScript is crunching numbers. A much better approach would be suck up all the rowset quickly and disconnect from the DB, then crunch numbers and finally dump the buffer to the client.

Consider using a disconnected recordset (you specify a client side static cursor) or even the simple GetRows method of the recordset object that dumps the whole rowset into a 2-dimensional array. This will mean that you maintain locks on the various tables for the smallest time possible.