Question

I am building a php site using jquery and the DataTables plugin. My page is laid out just as it needs to be with pagination working but in dealing with large datasets, I have noticed the server is pulling ALL returned rows as opposed to the 10 row (can be more) limit stated within each ‘page’.

Is it possible to limit the results of a query and yet keep say the ID numbers of those results in memory so when page 2 is hit (or the result number is changed) only new data is sought after?

Does it even make sense to do it this way?

I just dont want to query a DB with 2000 rows returned then have a ‘front-end-plugin’ make it look like the other results are hidden when they are truthfully on the page from the start.

Answer 1

The LIMIT clause in SQL has two parts — the limit and the offset.

To get the first 10 rows:

SELECT ... LIMIT 0,10

To get the next 10 rows:

SELECT ... LIMIT 10,10

To get the next 10 rows:

SELECT ... LIMIT 20,10

As long as you ORDER the result set the same each time, you absolutely don’t have to (and don’t want to) first ask the database to send you all 2000 rows.

To display paging in conjunction with this, you still need to know how many total rows match your query. There are two ways to handle that —

1) Ask for a row count with a separate query

SELECT COUNT(*) FROM table WHERE ...

2) Use the SQL_CALC_FOUND_ROWS hint in your query, which will tell MySQL to calculate how many rows match the query before returning only the 10 you asked for. You then issue a SELECT FOUND_ROWS() query to get that result.

SELECT SQL_CALC_FOUND_ROWS column1, column2 ... LIMIT 0,10

2 is preferable since it does not add an extra query to each page load.