Question

I am currently write large data to html for some periodic reporting reasons . It has just a table with about 10,000 rows . Which are getting large as 7MB . When users open it sometimes browser becomes unresponsive cause of large file .

I can store JSON on server and later on when people load page get that JSON and page through and show 1000 records at a time.

What is better approach to implement this ?

Answer 1

I make the following assumptions

1. the data is stored in a file
2. the data will only be appended to

If that is true, you can implement paging on the file. Use a random acces file. Simply read the file line by line for a number of records. Then when you are done, save the file position. When sending the html data to the browser also save the file position. This can be done using query parameter in a link to “get more data”.

When the user clicks the “get more data” link, you reopen the file, but first position() to the saved position. Then read the next few records, save the position etc.