Question

I have a reporting system where I need to combine data from 3 diffrent servers(systems) show it to users . This works fine as far as there few thousand records but as it increases system times out .

I was thinking to use another approach . When user runs result I will generate a request to generate report to database. A daemon running will pick it up and gather data and generate a report and will email to user a link to reporting data . I can do multiple things here .

1)insert gathered data to database in form where it can be simply displayed to users on page.
2)generate html and store it to database .

Although I think 1st option is best here are there any other suggestions ?

Answer 1

Several thoughts:

1. Consider using hadoop to make your report processing very scalable
2. Consider running the reports on an interval (e.g. every hour). When the user requests a report you can immediately show them the most recently generated report.
3. Definitely do not store html. Instead render it when the user makes the request. By the way, this also gives you the flexibility to show different versions of the reports to different users