Question

How can I have SQL repeat some set-based operation an arbitrary number of times without looping? How can I have SQL perform an operation against a range of numbers? I’m basically looking for a way to do a set-based for loop.

I know I can just create a small table with integers in it, say from 1 to 1000 and then use it for range operations that are within that range.

For example, if I had that table I could make a select to find the sum of numbers 100-200 like this:

select sum(n) from numbers where n between 100 and 200 

Any ideas? I’m kinda looking for something that works for T-SQL but any platform would be okay.

[Edit] I have my own solution for this using SQL CLR which works great for MS SQL 2005 or 2008. See below.

Answer 1

I think the very short answer to your question is to use WITH clauses to generate your own.

Unfortunately, the big names in databases don’t have built-in queryable number-range pseudo-tables. Or, more generally, easy pure-SQL data generation features. Personally, I think this is a huge failing, because if they did it would be possible to move a lot of code that is currently locked up in procedural scripts (T-SQL, PL/SQL, etc.) into pure-SQL, which has a number of benefits to performance and code complexity.

So anyway, it sounds like what you need in a general sense is the ability to generate data on the fly.

Oracle and T-SQL both support a WITH clause that can be used to do this. They work a little differently in the different DBMS’s, and MS calls them ‘common table expressions’, but they are very similar in form. Using these with recursion, you can generate a sequence of numbers or text values fairly easily. Here is what it might look like…

In Oracle SQL:

WITH   digits AS  -- Limit recursion by just using it for digits.     (SELECT       LEVEL - 1 AS num     FROM       DUAL     WHERE       LEVEL < 10     CONNECT BY       num = (PRIOR num) + 1),   numrange AS     (SELECT       ones.num         + (tens.num * 10)         + (hundreds.num * 100)         AS num     FROM       digits ones       CROSS JOIN         digits tens       CROSS JOIN         digits hundreds     WHERE       hundreds.num in (1, 2)) -- Use the WHERE clause to restrict each digit as needed. SELECT   -- Some columns and operations FROM   numrange   -- Join to other data if needed 

This is admittedly quite verbose. Oracle’s recursion functionality is limited. The syntax is clunky, it’s not performant, and it is limited to 500 (I think) nested levels. This is why I chose to use recursion only for the first 10 digits, and then cross (cartesian) joins to combine them into actual numbers.

I haven’t used SQL Server’s Common Table Expressions myself, but since they allow self-reference, recursion is MUCH simpler than it is in Oracle. Whether performance is comparable, and what the nesting limits are, I don’t know.

At any rate, recursion and the WITH clause are very useful tools in creating queries that require on-the-fly generated data sets. Then by querying this data set, doing operations on the values, you can get all sorts of different types of generated data. Aggregations, duplications, combinations, permutations, and so on. You can even use such generated data to aid in rolling up or drilling down into other data.

UPDATE: I just want to add that, once you start working with data in this way, it opens your mind to new ways of thinking about SQL. It’s not just a scripting language. It’s a fairly robust data-driven declarative language. Sometimes it’s a pain to use because for years it has suffered a dearth of enhancements to aid in reducing the redundancy needed for complex operations. But nonetheless it is very powerful, and a fairly intuitive way to work with data sets as both the target and the driver of your algorithms.