Question

Several colleagues and I are faced with an architectural decision that has serious performance implications: our product includes a UI-driven schema builder that lets non-programmers build their own data types for a web app. Currently, it builds properly normalized schemas behind the scenes and includes some complex logic to alter the schema and migrate legacy data automagically if the admins make changes to the data types.

The normalized schemas have hit performance bottlenecks in the past, and a major refactoring has been scheduled. One of the groups of developers wants to store every property of the data types in a separate table, so that changes to the data types will never require schema altering. (A single property could be turned into a 1:n relationship, for example, just by changing application logic.)

Because early benchmarking indicates this will exact a huge performance penalty, they have built a caching layer in application code that maintains denormalized versions of each data type. While it does speed up the queries, I’m skeptical about the complexity that the application layer will be taking on, but I’m hoping for feedback – am I being pessimistic? Have others deployed this type of solution successfully? Should I stick to my guns, or is moving the complexity from ‘Schema modification tools’ to ‘Schema mirroring tools’ a good thing?

Answer 1

The normalized schemas have hit performance bottlenecks in the past, and a major refactoring has been scheduled. One of the groups of developers wants to store every property of the data types in a separate table, so that changes to the data types will never require schema altering. (A single property could be turned into a 1:n relationship, for example, just by changing application logic.)

This sounds like a bad idea to me.

1. It’s going to mess up your database performance. If you store these things on one row they will be physically located together on the disk and treated as one thing for the purposes of locking, etc.
2. Queries that you write are going to require a mass of extra joins and will be very painful. You will end up writing views to turn it back into what it should have been in the first place.
3. The scenario described might never happen so you slowed down and complicated your application for potentially no benefit,
4. If it does happen and you are going to have to re-code and test a bunch of application code what’s the little extra effort in making a database change at that time? You can make your child table, copy the data down into it with an update, and drop teh column from the parent table
5. If you are successful, in the future a different application may attach to your database. They will not be able to tell what the real schema is, because that information is held by your application. Model data in the database.
6. The cache on your application server can get tricky if (a) there is too much to fit in memory, (b) you scale to multiple application servers, (c) you have a different application that connects to your database. You’re working around a performance problem that is of your own making.
7. You are not going to be able to create an index on multiple columns if they are each living in a child table.