Our set up today takes XML data and splits that data into multiple tables in SQL. The only benefit of this is reporting is good. However, whenever we want to retrieve data we are having to re-bind all the data from hundreds of tables to re-export the XML. Each XML could be several MB to several GB.
We hardly ever run reports ironically but do retrieve / save the data very often. Due to splitting it/compiling it with several tables, both saving and retrieval is not very efficient.
Since the data comes in as XML, I’m considering updating our method and saving the XML as a large BLOB into the table. That would be so simple.
The issue now comes with reporting – without the ability to index blobs I’m wondering what options I could have to run as efficient as possible reports.
The database is in the 100’s GBs.
I’m not a DBA (I’m a C# person) – I’ve just landed in this position at work so the only way I could think about this would be to do it using C# – build each BLOB as XML and then query the XML data in C#. This however, seems like it would be very inefficient. Maybe XQuery in SQL is better?! Despite not being a DBA, I’m more than happy for any programming (C#/VB) or SQL suggestions.
You can save the data in a single XML-type column in your database and then access the data via XQuery.
XQuery is for me, personally, a bit fiddly, but I found this list of tips to be of great help:
http://www.jackdonnell.com/?p=266
The advantage is that you are only persisting one version of the data so updates and reads are quick, apart from the XML parsing-bit (but that may depend on your data volume). Getting the data into the database from C# is straightforward, as you can map your XML to a corresponding
SqlDbType.