Question

When developing large systems (hundreds of tables) Do you create the indexes (and to lesser extend the other constraints in the DB) when you create the entities (tables), or wait for the system to be running (may be private Beta) to decide where to put the indexes?

Answer 1

I design indexes based on the eventual query scenarios. What will be the most common queries run against the table? That should inform index design – both to optimize query performance as well as to minimize insert/update/delete overhead.

Simply creating a clustered index on the primary key, for example, may make sense in a theoretical world up front, but may not mirror real-world query load.

For example: what if you have a table of order items, where 0-n order items are associated with a parent order? Do you just create an order item ID column, designate it the primary key, and burn your clustered index even though in the real world, 90% of your query activity against this table will be “get order items for order xyz”, implying that a clustered index on parent order ID might make more sense than the “default” primary key clustered index on order item ID?

You can do a lot of this up front by knowing what scenarios your application will enable. Then, you can also do traces in the real world and analyze them to find where you are missing indexes; SQL Server, for example, ships with tools to do this, there are third-party tools too. One technique I use sometimes is also to do a big trace, upload the trace info into a table, and query it for distinct SQL statements (based on whatever criteria… e.g. give me all UPDATEs against table xyz…) and then you can do a query plan for those statements and see how good your indexing is by, for example, looking for and addressing table or index scans appropriately – and verifying by re-examining the execution plan for the query.

Some cautionary notes… don’t apply indexes willy-nilly based on traces. An index on a table will affect overall performance of all queries against the table. Don’t assume that a table or index scan (rather than a seek) is necessarily bad; it doesn’t matter in a ten-row table. Index optimization is a combination of science and art, so keeping it simple is critical, testing frequently after small incremental changes is a good way to retain sanity and be able to roll back frequently, and above all, when you have a set of changes, script them out so that your DBA has an exact protocol of what will be done, and can easily determine where/what to roll back if needed.