Question

Currently, the ItemCollection#query method in the Ruby AWS SDK returns the whole dataset. I looked into the Ruby AWS SDK source code for any possibility of pagination, but the option closest to pagination is :limit, no pagination. According to the Amazon’s DynamoDB API (the HTTP, not Ruby) documentation for Query operation suggests there COULD be pagination with the response key LastEvaluatedKey:

Primary key of the item where the query operation stopped, inclusive of the previous result set. Use this value to start a new operation excluding this value in the new request.
The LastEvaluatedKey is null when the entire query result set is complete (i.e. the operation processed the “last page”).

So I can do paging by adding the :limit option, and then doing the next query for a range value after my last Item, but then I have no idea what the total count is unless I do a full query.

Is there a better/easier way of achieving pagination?

Answer 1

Depending on your needs, building a complete pager will likely require two requests, see Query and Scan in Amazon DynamoDB, in particular Count and ScannedCount:

The Amazon DynamoDB Scan and Query APIs use Count values for two
distinct purposes.

In a request, set the Count parameter to true if you want Amazon
DynamoDB to provide the total number of items that match the scan
filter or query condition, instead of a list of the matching items.

In a response, Amazon DynamoDB returns a Count value for the number of
matching items in a request. If the matching items for a scan filter
or query condition is over 1MB, Count contains a partial count of the
total number of items that match the request. To get the full count of
items that match a request, use the LastEvaluatedKey in a subsequent
request. Repeat the request until Amazon DynamoDB no longer returns a
LastEvaluatedKey.

[emphasis mine]

That is, to avoid complicated logic initially you’ll likely want to query the number of matching items upfront and request specific pages on demand thereafter only.

Good luck!