I’m using AWS’ SimpleDB as intended for the most part, but also want to store some crucial payment/financial data across several domains (tables). RDS is of course an option, but since I already have a decent layer over SimpleDB, if it were straightforward to implement at that layer, I figure why not.
So I guess, requirements:
- Lock out access to domain object(s) involved in the transaction
- Store previous versions of objects to facilitate rollback
- Detect an access to object(s) with a ‘broken’ or ‘failed’ transaction, and rollback everything involved
Since I can control all access to those domains, I think this is all possible. Is there a standard, obvious or common way to tackle this?
Off the top of my head:
- Add ‘current_transaction_guid’ attribute to all domains
- Each domain to have an audit/history domain storing previous versions (with IDs)
- When transaction starts, write to the ‘transactions’ domain with unique ID (say a GUID), start date/time, is_complete=false
- When writing changes to an object, store existing versions in history table
- Write changes across all domains, updating current_transaction_guid attribute in each case
- (After writes/updates done) update transaction object is_complete to true
- Update all domain objects current_transaction_guid to null
- Before reading/writing/bringing anything into a transaction, check current_transaction_guid. If null, go ahead, but if not null, read transaction status. If is_complete=false, wait for ‘true’ or until transaction timeout; if is_complete=true, wait a short time and then update current_transaction_guid to null
Ugh, come to think of it, maybe it’s not worth the hassle, and it would be better to pool all data for all domains involved into a single domain, even if massively redundant? Consistent reads and optimistic concurrency (via version number) should be enough for that?
I’ve designed a transaction system to use with SimpleDB. I’m not sure your scheme above will work due to eventual consistency. While consistent reads can be used, that will impact your performance and negate the scalability advantages of using SimpleDB (- probably better off with a cluster of RDBMS servers instead).
My mechanism doesn’t use consistent reads and makes use of the multi-valued items feature of SimpleDB to store a history of transaction,value pairs. Transaction IDs and status is stored in a transactions domain which also records which domain/cols a transaction effects.
Upon read, a consistent value of a col can be determined by constructing a tree where the nodes are values (current & historic) and the connections are version->prev_version dependencies and topologically sorting it. If there is a single tail (unique last element in the topological sort), then that is the consistent value for the column. Values belonging to uncommitted values are ignored in that process.
If there is no unique tail, that signifies a conflict. Conflicts are not possible to avoid in a distributed system – due to physics – is just isn’t possible to have a central clock against which to timestamp (or version count) values. That would required 0ms latency connections between all servers that make up the system (which is impossible – ask Einstein). Where conflicts are discovered, we pass that to the application to resolve (- since it sometimes doesn’t matter much and often can be resolved with additional information from the models).
We also timestamp transaction start and any transaction in the table not marked committed or aborted is assumed aborted if the timeout has elapsed.
Transactions and historic values can be pruned from rows at any point at which their was a unique node in all topological orderings.
Hope that gives you (or someone) some ideas if you decide to go in this direction.