Question

some more networking problems. I stumbled into this question: Serialization/Deserialization of a struct to a char* in C and a lot of the answers are making sense, but what I keep seeing over and over again is this ‘char data’ field that all packets seem to carry.

Now i know what it is for – you store the data you want to send in here. But how does one actually write data to it? Is there ways to just store entire objects in this data field? Or do i somehow serialize the object, store it in data, and then serialize the packet before i send everything off..?

Answer 1

Every variable exists in your computer’s memory. The memory is organized in bytes.

When you’re writing C++ code, you can directly read those bytes. For a struct, the memory for all of its members is in one contiguous chunk (although there may be gaps between each member).

So, if I declare:

struct foo {
    char x;
    char y;
    short z;
    int q;
};

Then when I create a struct foo, I get the following layout in memory (8 total bytes on most systems):

 xyzzqqqq

The first byte is x, the second y, the third and fourth together are z, and the last four are q.

So, the object is already “serialized” – you have a bunch of bytes which represent it. That’s all you need to send over the network: the information which represents the data structure.

The reason you’d write your own serializer is because you might want to change the way that the object is read or written (for instance, what if I added a field to struct foo?), because you need to communicate between machines where the memory layout is different (which byte of z represents the “most significant” portion of the number?), or because you only want to serialize part of the structure (what if we had some empty space between the members?).

But, fundamentally, the reason you’re sending “char data” is because everything in your computer can be represented that way. I’m not going into Turing’s proofs about symbol encoding, but it’s a mathematical certitude that any piece of knowledge can be encoded as a series of ones and zeroes.

In more concrete terms, the way you put data into the “char data” field of a packet is by memcpying from where the data currently are into the buffer. So if I had a char* target, I could write a struct foo x into it thusly:

memcpy(target, &x, sizeof(struct foo));

Or I could do it more carefully by writing each field:

memcpy(target, &x.x, 1);
memcpy(target+1, &x.y, 1);
memcpy(target+2, &x.z, sizeof(short));
memcpy(target+4, &x.q, sizeof(int));

The & is the address-of operator, if you didn’t already know. So I’m writing from the address of each member, into some offset within target, and writing a number of bytes equal to the length of the member variable representation.

The accepted answer to your last question pointed out that this is an oversimplification: when you send a multibyte integer over the network, you have to worry about the endianness (byte order). So what you actually do is this:

memcpy(target, &x.x, 1);
memcpy(target+1, &x.y, 1);
*((short*)(target+2)) = htons(x.z);
*((int*)(target+4)) = htonl(x.q);

This will handle reversing the bytes as appropriate to convert from host byte order to network byte order. Obviously the one-byte-long values are immune.