Question

So I’ve got this database that helps organize information for academic conferences, but we need to know sometimes whether an item is “incomplete” – the rules behind what could make something incomplete are a bit complex, so I built them into a scalar function that just returns true if the item is complete and 0 otherwise.

The problem I’m running into is that when I call the function on a big table of data, it’ll take about 1 minute to return the results. This is causing time-outs on the web site.

I don’t think there’s much I can do about the function itself. But I was wondering if anybody knows any techniques generally for these kinds of situations? What do you do when you have a big function like that that just has to be run sometimes on everything? Could I actually store the results of the function and then have it refreshed every now and then? Is there a good and efficient way to have it stored, but refresh it if the record is updated? I thought I could do that as a trigger or something, but if somebody ever runs a big update, it’ll take forever.

Thanks,

Mike

Answer 1

If your function is that inefficient, you’ll have to deal with either out of date data, or slow results.

It sounds like you care more about performance, so like @cmsjr said, add the data to the table.

Also, create a cron job to refresh the results periodically. Perhaps add an updated column to your database table, and then the cron job only has to re-process those rows.

One more thing, how complex is the function? Could you reduce the run-time of the function by pulling it out of SQL, perhaps writing it a layer above the database layer?