Question

I’m fairly new to Amazon’s AWS and its API for Java, so I’m not exactly sure what the most efficient method for what I’m trying to do would be. Basically, I’m trying to setup a database that will store a project’s ID, it’s status, as well as the bucket and location when uploaded to an S3 bucket by a user. What I’m having trouble with is getting a list of all the project IDs that have a status of “ready” under the status attribute. Any projects that are of status “ready” need to have their ID numbers loaded to an array or arraylist for later reference. Any recommendations?

Answer 1

The way to do this is to use the scan API. However, this means dynamo will need to look at every item in your table, and check if its attribute “status” is equal to “ready”. The cost of this operation will be large, and will charge you for reading every item in your table.

The code would look something like this:

Condition scanFilterCondition = new Condition()
    .withComparisonOperator(ComparisonOperator.EQ.toString())
    .withAttributeValueList(new AttributeValue().withS("ready"));
Map<String, Condition> conditions = new HashMap<String, Condition>();
conditions.put("status", scanFilterCondition);

ScanRequest scanRequest = new ScanRequest()
    .withTableName("MasterProductTable")
    .withScanFilter(conditions);

ScanResult result = client.scan(scanRequest);

There is a way to make this better, though it requires denormalizing your data. Try keeping a second table with a hash key of “status”, and a range key of “project ID”. This is in addition to your existing table. This would allow you to use the Query API (scan’s much cheaper cousin), and ask it for all items with a hash key of “ready”. This will get you a list of the project IDs you need, and you can then get them from the project ID table you already have.

The code for this would look something like:

QueryRequest queryRequest = new QueryRequest()
    .withTableName("ProductByStatus")
    .withHashKeyValue(new AttributeValue().withS("ready"));

QueryResult result = client.query(queryRequest);

The downside to this approach is you have to update two tables whenever you update the status field, and you have to make sure that you keep them in sync. Dynamo doesn’t offer transactionality, so you have to be ready for the case where the update to the master project table succeeds, but your secondary status table doesn’t. Or vice-versa.

For further reference: http://docs.amazonwebservices.com/amazondynamodb/latest/developerguide/QueryAndScan.html