Question

I’m developing a piece of embedded hardware that requires some data from a PC. I’m using a FAT32 formatted SD Card to provide this information. The data is just an excel file that I export to CSV. My question is, should I let the uC (AT Mega 128L) handle this using char* strtok (char *s, const char *delim) or should I write a small utility that converts this CSV file to a binary format?

Performance, as long as it’s reasonably fast, isn’t a major concern nor is the file size. The biggest issue would be SRAM usage.

The length of a line is max. 40 chars max with about. There are about 7 fields, two of which are basically indices. The uC is supposed to retrieve the information it’s interested in by looking through these indices and seeing if it matches. For example, suppose the uC is needs information regarding something which is stored at index 5. It then needs to go here and retrieve the other 5 fields and display them on a screen. The uC requires ‘random’ access to the file – i.e. at one moment, it may need something from index 7 and at another it may need something from index 70.

As I understand, it would be better if this was a binary file with a strictly defined format (i.e. each field would be a fixed number of bytes). The advantage would be that the uC could seek directly to the byte it was interested in. For example, suppose each ‘record’ takes up 100 bytes (it will be a lot less, but just as an example). The uC knows the 2nd index would start from 100 (0-99 for the first record), the third from 200 etc.

So if it needs to access the 7th record, it just seeks to the 700th byte and retrieves the relevant information. Would the binary file approach be better than CSV? My main concern is SRAM usage and reasonable performance.

Answer 1

The binary file approach will be better for a number of reasons. One of the big pieces that you didn’t mention is that you will need to convert the string data in the fields to numeric data (at least for the indices, if not for the other data), which tends to be expensive in both compute time and in RAM. In addition, strtok of course requires searching through strings, and if you don’t know while line you want until you look at it, that’s a lot of searching through strings and converting them to integers for each line you read.