Question

I work with a data logging system in car racing. I am developing an application that aids in the analysis of this logged data and have found some of the query functionality with, datasets, datatables and LINQ to be very useful, i.e. Minimums, Averages, etc.

Currently, I am extracting all data from its native format to a data table and post-processing that data. I am also currently working with data where all channels are logged at the same rate, i.e. 50 Hz (50 samples per second). I would like to start writing this logged data to a database so it is somewhat platform independent, and the extraction process doesn’t have to happen everytime I want to analyze a dataset.

Which leads me to the main question… Does anyone have a recommendation for the best way to store data that is related by time, but logged at different rates? I have approximately 200 channels that are logged and the rates vary from 1 Hz to 500 Hz.

Some of the methods I have thought of so far are:

• creating a datatable for all data at 500 Hz using Double.NaN for values that are between actual logged samples
• creating separate tables for each logging frquency, i.e. one table for 1 Hz, another for 10 Hz, and another for 500 Hz.
• creating a separate table for each channel with a relationship to a time table. Each time step would then be indexed, and the table of data for each channel would not be dependent on a fixed time frequency

I think I’m leaning towards the index time stamp with a separate table for each channel, but I wanted to find out if anyone has advice on a best practice.

For the record, the datasets can range from 10 Mb to 200-300 Mb depending on the duration of the time that the car is on track.

I would like to have a single data store that houses an entire season, or at least an entire race event, so that is something I am considering as well.

Thanks very much for any advice!

Answer 1

Can you create a table something like:

Channel, Timestamp, Measurement

?

The database structure doesn’t need to depend on the frequency; the frequency can be determined by the amount of time between timestamps.

This gives you more flexibility as you can write one piece of code to handle the calculations on all the channels, just give it a channel name.