Home/ Questions/Q 7794645
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-01T22:50:06+00:00

I’m currently working on some web services for a client of ours. Before we

  • 0

I’m currently working on some web services for a client of ours. Before we make it available to them we’d like to optimize performance of our database calls, as very LARGE amounts of data can potentially be returned. (could be tens of thousands of objects, could be millions, each object containing about 12 lists of other objects)

We don’t want to strain our servers, nor do we want to limit the web service unnecessarily.

One of the web service methods returns all data within a specified date range, I was thinking that if the amount of data being returned was larger than a set amount, return a message saying something like:

“Data too large, please reduce date range”

Is limiting the user’s scope like that a good idea??

I have to limit the amount of data our client can retrieve in one shot, but still keep at as convenient as possible for them. I mean, they’re programmers too so it doesn’t have to be too simple, but simple enough to use.

What are some good practices concerning returning large amount of data through a web service??

Thanks!

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You might be able to adapt the common technique of paging data displayed in a list or grid. The call to the database specifies the number of records to return, and the page number.

    So, for example, if they are displaying 10 records on a page, only 10 records will be returned for display. Records 1 – 10 (or 0 – 9, if you prefer) are returned for page 1, and 11 – 20 for page 2, and so on.

    Also often returned is the total number of records available.

    This way, the user can continue scrolling through a large number of records, or they can choose to refine their search criteria to yield a smaller resultset.

    You could consider this kind of paging or chunking approach for your web service. The web service call supplies the number of records to be sent in each chunk, and the “page” or “chunk” number. The web service returns the requested records, along with the total number of records available.

    With this approach, the developer who is consuming the web service remains in control.

    The calling code can be placed in a loop so that it continues requesting chunks, if that behavior is desirable. If someone really wants a truckload of records, they can just set the number of records argument to be a very large number (or you could make that an optional parameter, and return everything if it is null, zero, empty).

    • 0