Question

I have a small client server application in which i wish to send an entire structure over a TCP socket in C not C++. Assume the struct to be the following:

    struct something{
int a;
char b[64];
float c;
}

I have found many posts saying that i need to use pragma pack or to serialize the data before sending and recieveing.

My question is, is it enough to use JUST pragma pack or just serialzation ? Or do i need to use both?

Also since serialzation is processor intensive process this makes your performance fall drastically, so what is the best way to serialize a struct WITHOUT using an external library(i would love a sample code/algo)?

Answer 1

You need the following to portably send struct’s over the network:

• Pack the structure. For gcc and compatible compilers, do this with __attribute__((packed)).

• Do not use any members other than unsigned integers of fixed size, other packed structures satisfying these requirements, or arrays of any of the former. Signed integers are OK too, unless your machine doesn’t use a two’s complement representation.

• Decide whether your protocol will use little- or big-endian encoding of integers. Make conversions when reading and writing those integers.

• Also, do not take pointers of members of a packed structure, except to those with size 1 or other nested packed structures. See this answer.

A simple example of encoding and decoding follows. It assumes that the byte order conversion functions hton8(), ntoh8(), hton32(), and ntoh32() are available (the former two are a no-op, but there for consistency).

#include <stdint.h>
#include <inttypes.h>
#include <stdlib.h>
#include <stdio.h>

// get byte order conversion functions
#include "byteorder.h"

struct packet {
    uint8_t x;
    uint32_t y;
} __attribute__((packed));

static void decode_packet (uint8_t *recv_data, size_t recv_len)
{
    // check size
    if (recv_len < sizeof(struct packet)) {
        fprintf(stderr, "received too little!");
        return;
    }

    // make pointer
    struct packet *recv_packet = (struct packet *)recv_data;

    // fix byte order
    uint8_t x = ntoh8(recv_packet->x);
    uint32_t y = ntoh32(recv_packet->y);

    printf("Decoded: x=%"PRIu8" y=%"PRIu32"\n", x, y);
}

int main (int argc, char *argv[])
{
    // build packet
    struct packet p;
    p.x = hton8(17);
    p.y = hton32(2924);

    // send packet over link....
    // on the other end, get some data (recv_data, recv_len) to decode:
    uint8_t *recv_data = (uint8_t *)&p;
    size_t recv_len = sizeof(p);

    // now decode
    decode_packet(recv_data, recv_len);

    return 0;
}

As far as byte order conversion functions are concerned, your system’s htons()/ntohs() and htonl()/ntohl() can be used, for 16- and 32-bit integers, respectively, to convert to/from big-endian. However, I’m not aware of any standard function for 64-bit integers, or to convert to/from little endian. You can use my byte order conversion functions; if you do so, you have to tell it your machine’s byte order by defining BADVPN_LITTLE_ENDIAN or BADVPN_BIG_ENDIAN.

As far as signed integers are concerned, the conversion functions can be implemented safely in the same way as the ones I wrote and linked (swapping bytes directly); just change unsigned to signed.

UPDATE: if you want an efficient binary protocol, but don’t like fiddling with the bytes, you can try something like Protocol Buffers (C implementation). This allows you to describe the format of your messages in separate files, and generates source code that you use to encode and decode messages of the format you specify. I also implemented something similar myself, but greatly simplified; see my BProto generator and some examples (look in .bproto files, and addr.h for usage example).