Question

I am working on a StreamInsight application using StreamInsight 2.1.

The input adapter will be a live stock data stream with thousands of events per second.
I want to run multiple independent queries using the same data stream and be able to add more queries while the stream is running, picking up at the current location in the stream.

Whenever I write a query, though, it creates a brand new Input Adapter. So, I end up with 25 input adapters all loading and pushing the same data.

Am I just thinking about this wrong? Is an Input Adapter independent to each query?

Is the only solution to put the live stock data behind the Input Adapter and have each Input Adapter streaming that data again?

Any advice would be great, good documentation on StreamInsight is tough to find.

Answer 1

Since you are talking about input adapters, I’m assuming that you are using the legacy adapter model and not the newer Event Source/Sink approach. You need to take a look at what’s called “Dynamic Query Composition” or DQC. This approach will let you create a single instance of the input adapter and then share the stream with your other queries.

In a nutshell, you need to create a “Source” query that contains all your stock ticker data. From there, you can call “ToStream()” on the query to convert the results of the query back into a CepStream. Next have your 25 or so other queries/calculations/etc use that stream as their data source.

Take a look at Composing Queries at Runtime in the MSDN StreamInsight documentation. The first section on that page “Reusing the Output of an Existing Query” should be exactly what you need.