Question

I need to create TCP chat with C++ clients and Python server(already started), I have messages in c++ class like

class Message{
public:
uint64 utc_time;
uint64 token;
string content;
};

I am sending this from client to server, on server I have priority queue by utc_time and need to broadcast to others. My question is how to serialize this, which format to use so to avoid any cross language dependencies on size type size ? (maybe is going to be more meta data in future so need kind a little generic) ? Can anyone give me advice which format to use for serialization(or to flush only like bytes) ?

class Persistent:
public:
    Persistent(int sz):objSize(sz){}
    void write(std::ostream& out)const{out.write((char*)this, objSize);}
    void read(std::istream& in){in.read((char*)this, objSize);}
private:
    int objSize;
};

I thought of other possibility to have deserializator in c++ on server and call from python if that is possible. Any elegant solution to this problem ?

Answer 1

If you really want to go cross language and cross platform without having to worry about where the message ends, have a look at the combination of Google Protobuf and ZeroMQ.

When using regular sockets you would first read the size of the message (you would prepend this) and then you would know from where to where the byte-array is a complete message.

Example protobuf + zmq usage:

message Message {
    optional uint64 utc_time = 1;
    required uint64 token = 2;
    optional string content = 3;
}

Use the protobuf compiler to generate C++ code (or ruby/python/etc).

To use it in your code:

#include <Message.pb.h>

Message msg;
msg.set_token(1);
msg.set_content("Hello world");

To send it using zmq:

std::string serialized = msg.SerializeAsString();
zmq::message_t reply(serialized.size());
memcpy(reply.data(), serialized.data(), serialized.size());
zmq_socket.send(reply);

To receive it using zmq:

zmq::message_t request;
zmq_socket.recv(&request); // blocking
Message recv_msg;
recv_msg.ParseFromArray(request.data(), request.size());