Question

I’m planning to build an application that must allow the user to set up its own data model (i.e. create fields, data structure, etc.) dynamically.

I’m facing several technical possibilities, all having drawbacks. :

1. in admin screens, update the SQL schema of the DB to reflect the changes.
   • I fear this is a very bad idea because of the permissions the application must have on the DB. Moreover, if every click a new sql schema must be applied, I imagine I will run direct on a hole. This is the approach I’ve seen on most application customizable by the user.
2. create in DB schema a set of generic extra columns and hope there is enough columns for the complex data models.
   • this will quickly can be a functionnal limitation if I can’t allow more than X columns in my app
3. store in a single table all items with an ID column and a Xml column to store the user-defined columns.
   • this approach may remove the previously mentioned drawacks because the sql schema will remain static, but as EF (which I was hoping to be able to use) does not know how to manage xml data type, I will have to end up with either manual SqlCommand with XML function, or writing a custom EF provider, which I imagine will be quite a lot of work.
   • This the approach chosen by Microsoft for SharePoint… this let me think it’s the better one (or at least the less bad)
4. create a “properties” table with basically a itemId column, a property name column and a property value column
   • this approach implies a very very large table (X items * Y properties per item)
   • I will have to store my values in plain text, even if it’s numerical for example.

My requirements are :

• keep the code maintainable, unit testable and all fashioned technics
• have a responsive application with large amount of data
• have an as secured as possible application
• allow the users to fully customize their application (create custom view with filter / sort on user properties).

I feel the choice of the correct design must be the good one now, because it would be very hard to change this latter.

Any feedback would be appreciated

Answer 1

One option would be to use a NoSQL database like MongoDB which is schema-less. New fields don’t need to be defined up front (no schema modification headaches) and different records can have different fields. This is one of the benefits of a NoSQL store like this.

e.g. in mongo, your “table” could have these 2 records within in legimitely:

{
    "ID" : 1,
    "FirstName" : "Joe",
    "LastName" : "Bloggs",
    "FavouriteColour" : "Blue"
}

{
    "ID" : 2,
    "FirstName" : "John",
    "LastName" : "Smith",
    "DOB" : "2000-01-01"
}

Adding in a new field is as simple as just start including it in records.

In my experience, having a fully flexible/dynamic schema in a RDBMS like SQL Server can be a bit of a pain and be challenging to achieve high performance. I’ve had experience with options 1) and 3) that you listed. When data was stored as XML, I ended up usually needing to shred the data out into relational form anyway for certain purposes.