Question

I need help in the following design. I expose a web method that does an SQL select from a database. The problem is that the number of records can be huge and I don’t want to return all of the records in a single call.
So I can think of these options (to return the result in pages):
1) Provide a method with parameters so that the client requests recordStart and recordEnd each time.
2) Modify the method to accept a resultset of size X and somehow understand that each request is not a new one but return the next X records. To figure this out somekind of a token could be associated per client but the problem is I am not sure how long should this token be kept and then disposed of so as to treat an incoming request either as a first request or a continuation of a previous one.

So which design should I go for and how would I solve any relevant problems I mention?
Are there better ways to deal with these problems?

Answer 1

From my point of view:

• each request may be encapsulated in a request object that basically will hold an offset and a page size;
• each response may be encapsulated in a response object that basically will have a result list and a total, alternatively you could hold the request object used to build the response;

Your interface to perform the selection on database will be similar to:

public PageResponse getPage(PageRequest pageRequest);

This approach turns easy the extension of your paging method, imagine in a few months you need to implement a sort to that method, you will need to change each invocation to it. With this approach you change the PageRequest object and gives a default sort, nothing will be broken and you can customize sort just in the invocation that really needs it.

Within this method you will need two different DataBase selections:

• one in order to retrieve the selection list (the one that will be hold by the response and accessed through the property resultList), this can be done using feature specific for each database to limit your result set (top for sybase, limit for mysql and PG, rownum with Oracle, this will vary from one database to other);
• another one to get the total of selected records without paging, in order to perform paging of your data in case of big data sets.

A good reference for your problem would be Spring Data, they have Page and PageRequest that is more or less what you need. Maybe you could use their API to implement your solution.

---

Practically your request object could looks like:

public class PageRequest {

    private int offset;
    private int pageSize;

    // getters and setters and convenience constructors with the given fields

}

public class PageResponse {

    private List<?> resultList;
    private int total;

    // getters and setters and convenience constructors with the given fields

}

Of course you could play a bit with Generics too in order to have response holding types you already requested, facilitating use of the response object like:

public <T> PageResponse<T> getPage(PageRequest<T> pageRequest);

having the objects for Request and Response like:

public class PageRequest<T> {

    private int offset;
    private int pageSize;

    // getters and setters and convenience constructors with the given fields

}

public class PageResponse<T> {

    private List<T> resultList;
    private int total;

    // getters and setters and convenience constructors with the given fields

}