I have a program that captures live data from a piece of hardware on a set interval. The data is returned as XML. There are several things I would like to do with this data (in order): -display it to user -save it to disk -eventually, upload it to database
My current approach is to take the XML, parse it into a hashtable so I can display the correct values to the user.
Next, I want to save the XML to a file on disk. For each data capture session I am planning on creating a unique XML file and I will dump all the data into it.
Finally, I would like to reparse the XML and upload it to a MySQL database. The data cannot be immediately uploaded to the database.
This seems really inefficient method of solving this problem and I would love some advice.
Is it a waste of hd space to save the data as XML?
Is it THAT inefficient to have to reparse the XML in order to write it to a database?
Thank you!
To clarify: a typical XML response will be ~1kb and are captured at a rate of about 1 response every 15-60 seconds.
I think I do want to store the XML as XML on the disk because the data is very valuable and a pain to reproduce (if it is even possible). Thank you!
When you receive a new XML document from the source, directly save to disk and parse it to display to the user.
With a background process, or user initiated, read the xml files from disk and send them to the server based on created date (so you can retrieve only the latest ones) for insert into MySql.