In a Linux embedded application I’m developing, there is the need to record some events that happen from time to time. These records are saved on a MTD flash device and once written there is no need to change them or do efficient searches, but read access is required to show the data back to the user. A big problem is that power can go away at any time, without a proper shutdown sequence. The frequency these events occur can be very slow (days/weeks), but several of them will occur at once. The data to be saved for each event is strongly typed: date, time, a couple of short text strings and several integers.
Currently I inherited a solution based on jffs2 and SQLite that is far from optimal because the DB file sometimes get corrupted. When this happens the whole file gets unreadable and there is no way to understand if it was caused by a bug in jffs2, or in SQLite or if the flash sector was bad, or if the power was cut at the wrong time.
Is there a library or a combination of filesystem/library that can better help me solve this kind of problem ? Or should I just use a text file with a CSV-like format ?
We are using plain old syslogd to a YAFFS2 partition on NAND flash, it appears to work well: when messages are sent to the logger and power is removed immediately after (<100ms) the message is there and the log never appears to corrupt.
This is based on observation rather than my explicitly knowing that everything will always be consistent by design, mind.