Question

So, I’m getting this data. From the network socket, or out of a file. I’m cobbling together code that will interpret the data. Read some bytes, check some flags, and some bytes indicate how much data follows. Read in that much data, rinse, repeat.

This task reminds me much to parsing source code. I’m comfy with lex/yacc and antlr, but they’re not up to this task. You can’t specify bits and raw bytes as tokens (well, maybe you could, but I wouldn’t know how), and you can’t coax them into “read two bytes, make them into an unsigned 16bit integer, call it n, and then read n bytes.”.

Then again, when the spec of the protocol/data format is defined in a systematic manner (not all of them are), there should be a systematic way to read in data that is formatted according to the protocol. Right?

There’s gotta be a tool that does that.

Answer 1

You may try to employ Boost.Spirit (v2) which has recently got binary parsing tools, endianness-aware native and mixed parsers

// This is not a complete and useful example, but just illustration that parsing
// of raw binary to real data components is possible
typedef boost::uint8_t byte_t;
byte_t raw[16] = { 0 };
char const* hex = "01010000005839B4C876BEF33F83C0CA";
my_custom_hex_to_bytes(hex, raw, 16);

// parse raw binary stream bytes to 4 separate words
boost::uint32_t word(0);
byte_t* beg = raw;
boost::spirit::qi::parse(beg, beg + 16, boost::spirit::qi::dword, word))

UPDATE: I found similar question, where Joel de Guzman confirms in his answer availability of binary parsers: Can Boost Spirit be used to parse byte stream data?