I have a custom python script that monitors the call logs from a Nortel phone system. This phone system is under extremely high volume throughout the day and it’s starting to appear that some records may be getting lost.
Some of you may dislike this, but I’m not interested in sharing the source code or current method in any way. I would rather consider this from a “new project” approach.
I’m looking for insight into the easiest and safest way to reliably monitor heavy data output through a serial port on Linux. I’m not limiting this to any particular set of tools or languages, I want to find out what works best to do this one critical job. I’m comfortable enough parsing the data and inserting it into mysql that we could just assume the data could be dropped to a text file.
Thank you
Well, the way that I would approach this this to have 2 threads (or processes) working.
Thread 1: The read thread
This thread does nothing but read data from the raw serial port and put the data into a local buffer/queue (In memory is preferred for speed). It should do nothing else. Depending on the clock speed of the serial connection, this should be pretty easy to do.
Thread2: The processing thread
This thread just sleeps until there is data in the local buffer to process, then reads and processes it. That’s it.
The reason for splitting it apart in two, is so that if one is busy (a block in MySQL for the processing thread) it won’t affect the other. After all, while the serial port is buffered by the OS, the buffer size is limited.
But then again, any local program is likely going to be way faster than the serial port can send data. Serial transfer is actually quite slow relative to the clock speed of the processor (115.2kbps is about the limit on standard hardware). So unless you’re CPU speed bound (such as on an Arduino), I can’t see normal conditions affecting it too much. So your choice of language really shouldn’t be of too much concern (assuming modern hardware). Stick to what you know.