Question

I have an ASP.NET data entry application that is used by multiple clients. The application consists of multiple data entry modules that are common to all clients.

I now have multiple clients that want their own custom module added which will typically consist of a dozen or so data points. Some values will be text, others numeric, some will be dropdown selections, etc.

I’m in need of suggestions for handling the data model for this. I have two thoughts on how to handle. First would be to create a new table for each new module for each client. This is pretty clean but I don’t particular like it. My other thought is to have one table with columns for each custom data point for each client. This table would end up with a lot of columns and a lot of NULL values. I don’t really like either solution and suspect there’s a better way to do this, so any feedback you have will be appreciated.

I’m using SQL Server 2008.

Answer 1

As always with these questions, “it depends”.

The dreaded key-value table.

This approach relies on a table which lists the fields and their values as individual records.

CustomFields(clientId int, fieldName sysname, fieldValue varbinary)

Benefits:

• Infinitely flexible
• Easy to implement
• Easy to index
• non existing values take no space

Disadvantage:

• Showing a list of all records with complete field list is a very dirty query

The Microsoft way

The Microsoft way of this kind of problem is “sparse columns” (introduced in SQL 2008)

Benefits:

• Blessed by the people who design SQL Server
• records can be queried without having to apply fancy pivots
• Fields without data don’t take space on disk

Disadvantage:

• Many technical restrictions
• a new field requires DML

The xml tax

You can add an xml field to the table which will be used to store all the “extra” fields.

Benefits:

• unlimited flexibility
• can be indexed
• storage efficient (when it fits in a page)
• With some xpath gymnastics the fields can be included in a flat recordset.
• schema can be enforced with schema collections

Disadvantages:

• not clearly visible what’s in the field
• xquery support in SQL Server has gaps which makes getting your data a real nightmare sometimes

There are maybe more solutions, but to me these are the main contenders. Which one to choose:

• key-value seems appropriate when the number of extra fields is limited. (say no more than 10-20 or so)
• Sparse columns is more suitable for data with many properties which are filled out infrequent. Sounds more appropriate when you can have many extra fields
• xml column is very flexible, but a pain to query. Appropriate for solutions that write rarely and query rarely. ie: don’t run aggregates etc on the data stored in this field.