Question

I figure there has to be a specific design reason why you can’t write a query like the following one:

select 
    (select column_name 
    from information_schema 
    where column_name not like '%rate%' 
    and table_name = 'Fixed_Income')
from Fixed_Income

and instead have to resort to dynamic SQL.

Anyone knows what that reason is? I tried Googling it, but all the hits were cries for help in solving the problem — meaning it’s a pretty widespread need and not well understood.

Answer 1

You ask a very interesting question.

The “relational” in “relational algebra” refers to name-value pairs, not to relationships between tables. In relational algebra, there is no requirement that all records in a set (table) have the same columns.

My best guess is that the limitation is related to the idea of entity-relationship diagrams comes into play. A database is designed around tables, and these tables have relationships to each other. The choice of a relational database for data storage and access was specifically when the data could be stored this way. Knowing the entities and their attributes suggests a static form of the data and hence static references in queries.

In addition, SQL as a language is a declarative language rather than a procedural language. This suggests — but does not impose — a compilation step separate from the running of the query. In general, the SQL engine does the following (at a very high level):

1. Compiles the query, generally into some sort of data flow process.
2. Optimizes the data flow process. (Typically part of the compilation process.)
3. Runs the query.

The first two result in what is called “the query plan”. You really cannot do optimization, though, unless you know about the objects you are operating on. So, dynamically choosing tables and columns means that optimization would be part of running the query rather than compiling it.

Finally, some databases like SQL Server support dynamic SQL. This allows you to build strings that get compiled and run at the same time. This is very useful for complex decision support queries. It is not recommended when you need fast transaction throughput, because the overhead for compilation is too high relative to the query.