Question

Generally speaking, for combining a lot of data is it better to use a temp table/temp variable as a staging area or should I just stick to “UNION ALL”?

Assumptions:

• No further processing is needed, the results are sent directly to the client.
• The client waits for the complete recordset, so streaming results isn’t necessary.

Answer 1

I would stick to UNION ALL. If there’s no need to do intermediary processing, thus requiring a temp table, then I would not use one.

Inserting data into a temp table (even if it’s a table variable which despite the myths, is not a purely “in memory” structure) will involve work in tempdb (which can be a bottleneck). To then just SELECT * as-is and return it without any special processing is unnecessary and I think bloats the code. When you just need to return data without any special processing, then a temp table approach seems a bit “round the houses”. If I thought there was a reason to justify the use of a temp table, I would run some like-for-like performance tests to compare with vs without temp tables – then compare the stats (duration, reads, writes, CPU). Doing actual performance tests is the best way to be as confident as possible that whatever approach you choose, is the best. Especially as you don’t have to be using temp tables for there to be work pushed over into tempdb – i.e. depending on your queries, it might involve work in tempdb anyway.

To clarify, I’m not saying one is better than the other full stop. As with most things, it depends on scenario. In the scenario described, it just sounds like you’d be adding in an extra step which doesn’t seem to add any functional value and I can’t see you’d gain anything other than creating a slightly more complicated/lengthy query.