Question

I am looking for design guidance here more than actual solutions, but welcome both.

The scenario is that I have a publisher which is responsible for publishing user updates performed in a system. My downstream systems are the subscribers to these updates.

The challenge I am having is that the users of the upstream system are regularly saving their work. The reason they are doing this is because one “logical” update actually involves transitioning many screens plus it is also natural for them to be multitasking on other things and as such they hit “save” a lot. Each time they hit save we receive a message.

So For each “logical” update we have maybe 5-10 individual update messages, some of which could be duplicated.

This causes overhead for the users of my downstream systems, who are being overwhelmed with the volume of updates. For each update they need to first check if this represents an “actionable” piece of work, and discard it if this is only the result of an upstream save.

I know the best solution would be to change the upstream system to batch the saves together into a single update but this would be too costly as this would have to involve the vendor.

Edit: There are no sequencing data in the messages. So when we receive the update it there is no way of telling how “far” through the process the user is. They could have finished it all in one go, or they could only be 10% of the way through and we will get 10 more updates before they are finished.

Edit: specifically I am using BizTalk as the messaging platform.

Edit: The pattern I want to implement is the Aggregator – http://eaipatterns.com/Aggregator.html. The problem is I have no way of knowing when the series of messages which comprise the input is complete or not.

Answer 1

I decided to implement a variant of the aggregator pattern whereby the default message (the most recent) is chosen from the collective interchange.

However, for the “trigger” (the set of conditions which indicate that the aggregation is complete), rather than rely on the completion of the series I will have a service window set at a certain time each day.

Hope this helps others.