Question

I’m working on a Gui-Module for Qt to plot realtime measurements like in an digital oscilloscope, based on Qwt. Everything works so far like it should, but maybe there are some features left to add 😉

In the moment the data is stored column-wise in QVectors, together with one global timeReference QVector in one seperate QObject. So the data can be discarded row-wise to keep only Meusurement up to a certain past. All QVectors always have the same length. The complete data can then be row-wise time-correctly plotted in a QwtPlot.

I wanna encapsulate the data-storage more, to be more independent on working with the measurements. Therefore it would be nice to add a seperate List of time-coordinates for every measurement and put them both in a seperate QObject, which accepts and delievers the data. Then there would be 10 or 20 of such QObjects, one for every data-channel, which are seperately plotted by an overlying QObject on the QwtPlot.

The data can now be dynamically whatever — how the data is stored, altered or discarded in between should not be visible to the outside.

My Question is: Is this smart? 20 or 30 QObjects containing each 10000 measurements, 10000 time-values, plus a seperate memory region of similar size (dynamically filled) where a subset ob the data is presented for plotting…? Is it sane to receive each measurement in its QObject as a signal, firing at around 1kHz? The signal/slot-part comes from the idea of making every OBject a QThread later, and implement real-time filtering, like low-pass or FFT on the data — therefore, signal/slot connections are handy to controll the output in an multithreaded environment?

How could the data be stored efficiently inside my OBjects? I’m thinking about two QList, one for time and one for precious data. Then allocating two plain double-arrays on the fly for dynamic access, whose pointers together with the length are put in a struct and returned by an accessData(pastTime) method. The dynamic memory is filled with the timeVal/measurement combinations from “now” up to a certain point in the past, settable by a signal. Everything fragile secured by mutexes inside the QObject.

When discarding old values, the QList has to be searched from the beginning for the first value which is young enough to be kept, the ones residing before this index are thrown away. Is a QMap smarter because of its upperBound() function? I figure that the hidden overhead wouldn’t be worth it.

How would one pro try to solve this problem nicely, efficiently or without any hassle? Special Qt-Features I should know about? Or are even free solutions out there? Anyhow, much text for such a basic question… Thanks for reading up to here 😉

Thanks in advance

Marvin

edith: did some cleanup in argumentation after stijns comments.

Answer 1

photo_tom’s answer pretty much sums it up: I’d stay away from QObjects for implementing the data handling and processing.

• should you ever decide to use something else than Qt for your gui, you’ll have a much harder time refactoring the code. Classes like QList and QVector can be replaced by STL counterparts without much problems, but the signal/slot part is something else.
• any 3rd part implementation for signal processing like filtering/fft is likely to take raw pointers to 1D or 2D data, so you’ll have to get those out of a QVector which I’m not even sure is possible. If it’s not, you’ll have to get each sample out of the QVector and copy it to a memory chunk, then process, then put it back in the QVector.
• which brings us to your question about QList/QMap: it can be done with any of them but they are actually designed as dynamic containers with random access iterators, while you are having chunks of memory of a fixed size holding 2D data. It might be worth looking into a custom data container class that covers your needs exactly. Take a paper, write down what you actually need, clear your mind and forget about Qt/STL/… then think about what components you’d need to implement this, then go further and think about how these components can be implemented (eventually in terms of Qt).
• for code like this it’s best not to reallocate, ever. Set limits on beforehand of the max history and number of samples you’re going to allow (or make it a configuration setting), and allocate the required arrays at the start of the program, reusing the same memory further on.
• consider a circular buffer. It is convenient when data comes in at another rate than it’s taken out (as is mostly the case with data acquisition), it needs no reallocations, it automatically keeps a history and can be implemented with a minimum amount of memory copies if it’s read/write methods return pointers to the underlying memory directly.
• maybe rethink your thread idea, it might make your code needlessly complicated: in the end, all data from the different channels has to make it to the screen at the same time. If the user is seeing data from channel 1 at time x, the data from channel 2 must also be from time x, else it doesn’t make much sense as a scope. But suppose you process the data from those channels in different threads, you need extra synchronization in the thread that does the actual display as not all data threads will finish processing the blocks at time x at te same time. Apart from that, consider that there might be no performance gain whatsoever: if a cpu has to calculate an FFT for 30 channels and clips at 100%, is it really going to calculate those FFTs faster if they are split over 30 threads?
• btw you say this is a basic question, I think it’s not actually. I’ve done a number of applications for data-acquisition/processing/visualisation/saving on different devices, and I consider those apps he hardest to develop..