We import data from our clients into our central server for reporting purposes. We use a .net 2 windows service installed on the client machines that sends the data extracted from the application to a WCF service as XML. We cannot go above .Net 2 on the client windows service.
We store the XML in a SQL server table, XML column. We then scrub the data and insert it into the required tables.
However the xml uses the column names of the data source that is quite large. This has therefore made the XML quite chucky. At the moment we are using the column names as node names and the column data as node values. This results in the column name being used as an opening and closing tag.
We have thought of using the attributes of XML but our concern is the use of “ within the data we receive. But again the column names are quite large and this still creates a fair amount of over head. So an idea bouncing around is using aliases for the column names and using the attributes of the XML. This should reduce the overhead significantly.
What I was wondering is what other people are doing to get around these kind of issues? Also are there any suggestions or best practices with regards to this kind of thing?
I wouldn’t corrupt the XML at all.
There is inherently some (slight) compression on the XML data type by SQL already.
However, if you absolutely need to minimize the data (without worrying about it being decypherable without first decompressing), then would look at compressing it using a Zip or Lzh type algorithm and then storing it in a varbinary – this might help.