Question

I have an EventsManager that receives events from an external source. An Event has a type and a value.

Listeners can be registered to the EventsManager to be informed of the succesive values of a certain type of event.

The EventsManager promises two things, for a given type of Event:

• the same value will not be sent twice in a row (when listeners get notified, they are guaranteed that the value they receive is a different value from the previous notification).
• the order in which values are received from the external source must be preserved, for a given type of event.

I have a working synchronized version but I would like to improve the throughput.

Typical use: < 1k listeners, < 10k event types, < 1M events received per second (but most are discarded because there is no listener registered for that type of event or the value has not changed).

• What would be the most efficient strategy to implement that behaviour (for example I could use one queue / lock per event type and hold them in a ConcurrentMap but having 10k queues doesn’t sound like a good idea)?
• Are there any existing libraries that would do something like that using scalable concurrent structures?

Example: Listener lst1 wants to listen to events of type type1
The EventsManager receives:

event: type2, value: 2
event: type1, value: 1
event: type1, value: 1 //no change => discard
event: type3, value: 4
event: type1, value: 7

lst1 should receive, in that order: 1 (only once) then 7.

Answer 1

I would try to implement this event flow

1. All incoming events are put into an initial EventQueue
2. An EventDispatcherThread reads the EventQueue, and filters and routes the events into appropriate EventQueue for each type (simple map of queues)
3. Multiple instances of EventListernerThread are reading the appropriate EventQueue of its type…

no locks/synchronisations are needed