Question

I have a task to convert the data design in XML format to a SQLite DB format. The criteria is to achieve more performance while keeping the same level of data scalability as with XML data.
The problem I face is that the XML data format can be divided based on the independent nodes which can be created as separate tables in the DB. However if the number of child nodes in the independent nodes increases, there will be an increase in the number of columns in the table. I found that there are 500-600 columns on an avg for a table which might seriously impact the query performance.
The data rows need to be frequently updated & queried for reading. Performance is the sole criteria for this change keeping the CPU load to minimum for that process.
Please share an idea to get around this problem. Ideas for converting the XML data to SQLite DB format are most welcome.

Thanks in advance.

Answer 1

If all columns are device parameters or similar you can create a table with a ParameterName column and a ParameterValue column.

DeviceId ParameterName ParameterValue
-------------------------------------
1        FrameRate     60
1        Brightness    75
1        BootTime      20
2        FrameRate     30
2        Brightness    100
2        BootTime      10

etc

If your parameter names are the same for many devices you should break out ParameterName to a separate table

ParameterId ParameterName
-------------------------
1           FrameRate
2           BrightNess
3           BootTime

and insert ParameterId in your main table instead of ParameterName.
This will improve performance and reduce storage space.