Is there a good way to determine how many pages the twitter search api has returned or is there a way to determine how many values were returned and divide that by the number of twits per page?
Share
Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
Is there a good way to determine how many pages the twitter search api has returned or is there a way to determine how many values were returned and divide that by the number of twits per page?
No. The API does not expose this; not because it’s not a useful feature, but because of the performance aspects of providing it.
In order to get a complete count of results, it is necessary for the search algorithm to completely iterate its index for each query. Then when you went back for the second page, it would have to iterate its index from page 2 onward to give you the count again. This means that getting all the data would be O(n^2) (because returning each of the N pages requires scanning all the later pages) instead of the expected O(n).
Because most requestors only want a few pages of results, it’s a common optimization for the query to return only partial results, with just a pointer into the index to allow the search to continue at the point it left off.
Most high-scale paginated APIs behave in a similar fashion for these reasons. To get an accurate count, you’ll have to force the query to completely iterate its index by looping through the pages. This comes with a high cost to the remote service, and making you come back many times allows the service to appropriately throttle your query so it does not negatively impact other users.