I created some scripts which generate the data at intervals. Say, for each hour some set of data is generated. The generated data is represented in the form of text/XML. I haven’t chosen the format to store. But currently I am thinking as XML. If at each hour an XML file generated based on test results, I want to store the hour instance in a databse table row. If it is TXT format, then i have to parse the document. The better would be an XML. If I add the each horu instance as new node to XML, end of the day, I want to move the complete day result to database. If not, each hour instance I have to make a new XML file and push the data into database. Can you suggest me the best way of doing it. I am confused whether to push the data at hour instance or end of the day. I don’t have any supported API in the scripting to connect database directly. I am planning to write a java program and trigger it end of the day to push the complete day data into database. I hope this topic would be intresting to others. 🙂
Share
The description of the task sounds like gathering log data and storing into a database. The decision to push hourly or daily should be taken upon the need to query the database later. If you want to create daily reports on the collected data – it might be OK to store it as 1 row = 1 day. But if you want to get reports by hour – then storing 1 row = 1 hour is better approach, otherwise you’ll have to additionally parse the data to extract the hour usage.
Anyway it could be hard to process stored XML later(although there are database servers to support that). If this is really a log data database, a NoSQL solution could be better in this case.