Home/ Questions/Q 6686679
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-26T05:11:59+00:00

I’m looking for a best practise way to handle incoming time series data. One

  • 0

I’m looking for a “best practise” way to handle incoming time series data.

One data point consists for example of time, height, width etc. for every “tick”. Is it a good idea to save n data points in-memory with a collection class and later “flush” the points to a database after reaching the limits of the collection?

Or should the data points be directly written to the database in the first place, so that my object can run queries against it?

I know that this is little information about my requirements, so the question is how fast is the data access to a database compared to a hybrid in-memory and database solution.

Say there are at most 500 data points per second to handle and the data has to be calculated somehow on every point incoming. With a pure database solution, one has to run a store query on every incoming point. I guess this is not effective, but I don’t know if such a database is able to “listen” and do this fast.

A nice feature for the database would be to send the points to subcribers. Is this possible with SQL server?

Thanks, Juergen

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Putting the “sending to subscribers” requirement aside, don’t get into the trap of premature optimization.

    I would try the simplest solution first, which is probably just writing the data into the database as it arrives. Then run stress tests. If the performance isn’t up to scratch, find the bottlenecks and optimize them out.

    Turning to the “sending to subscribers” requirement, this isn’t really something which relational database platforms are typically designed for (they are more about storing data and exposing it for on-demand retreival). A pub-sub type requirement is usually best solved using some kind of message bus. Perhaps take a look at something like NServiceBus.

    • 0