Question

I have an application which multicasts certain packed POD structs and a listener service which runs in other binaries. The listener service knows what the structs look like so when it receives them it casts it back into that struck with a reinterpret_cast and performs a callback.

The problem down the line is if binaries are released and new information needs to be added to the structs, those binaries would have to be rebuilt or they will be reinterpret_cast – ing and misusing information. This can be a problem in a production environment where one doesn’t have that flexibility all the time.

One thing I was told was that the way to go about it is to introduce the new style message, and have both of them being sent out..and with time, applications will eventually switch over to the new type of binary message until one can stop sending the old one. I was wondering if there is a better alternative.

For example, if one makes the convention of only adding new fields to the end of the packed struct, then old listening binaries might still be able to get to those new fields if they want to, and the ones built with the old information might still be able to access the top portions. So for example, if the sender is multicasting this:

struct foo {
    int  a;
    char b[2];
} __attribute__ ((packed));

and then several binaries are built on receiver ends which get const char* msg messages on the wire and do this:

foo* fooPtr = reinterpret_cast<foo*>(msg); 
registeredGuy->callback(fooPtr);

Now if we ever decide to roll out extra information on the sender side, old listeners might ok if we just tack it to the bottom like so:

struct foo {
    int  a;
    char b[2];
    char newStuff[17];
    int  k;
} __attribute ((packed));

old receivers should be able to still successfully cast and access their old info, while new guys can access the new stuff. Is this true? And are there better solutions which do not induce a hit on the speed (performance is very critical)

Answer 1

Sending two versions of what is essentially the same message is a bad idea. Especially for performance-critical systems. You end up spending at least twice as much time doing the actual sending. You also end up broadcasting more than twice as much information, so you could saturate your network.

The problem of versioning message payloads has existed as long as mesasge payloads themselves. My way of thinking is that the best way of fixing the problem is avoiding it altogether.

The key is understanding how the clients will receive and process the inbound data. Typically a client is going to listen for a UDP frame on the wire, suck it in, and processes that frame as a message. Ideally, your UDP frames are smaller than the MTU for your architecture (say 1500 bytes), so the messages won’t get chopped up in transit. They may arrive out of order, but that’s an altogether different problem.

Clients know how big the UDP frame was, because they pulled it off the wire. They also know how big the message they will process is, because it’s just sizeof(MessageType). The only thing they don’t know is the difference between the size of the frame and the size of the payload. You can tell them that, by including a fixed-size header with every message.

The header would look something like this:

struct MsgHeader
{
  size_t  msg_size_;
  int  msg_type_;
  char payload_[0];
};

The actual message would either overlay this, or come immediately after it (in &payload_[0]).

The client now reads the UDP frame, gets the size of the frame from the header, and pulls that many total bytes as a single mesasge. Starting at the payload pointer, the client casts the inbound data as the message type in question. If there is more data in the single message than fits in the message type the client understands, the client just ignores it and drops it on the floor.

As you make changes to the mesasges, you need to ensure backward binary compatibility, by not moving the location of any existing fields. Add your new fields at the end, increase the frame size in the header accordingly, and Bob’s your uncle.