Question

I have played a while with ZeroMQ and have couple questions/problems that I came up with. Would appreciate if any contributer to ZeroMQ could chime in or anyone who has used or currently uses the library.

* Let’s say I have one router/forwarder and 2 different clients(c1,c2). I want to push messages from client1 to client2 through the routing device. The router pulls messages from whichever client (here client1) and publishes them to any subscribed client (here client2). I currently the only way to route such messages to the appropriate client is through pub/sub, however , a) I want to decide how to route at runtime by sending a routingTo tag along with the message body, b) I want to use push/pull to forward to clients, not pub/sub because I want to implement blocking functionality when setting the high water mark property, c) I want to have c1 and c2 connect on exactly 1 port for pushing and 1 port for subscribing. Can I somehow make changes on the router side in order to not having to use pub/sub or is pub/sub the only way to route to clients even I know on the routing side where a message is supposed to be forwarded to? I read that pub/sub drops messages when queue size exceeds the hwm which I dont want. I also do not want to implement the request/reply patters because it adds unnecessary overhead as I do not need replies.

* After running below code (Push/Pull -> Pub/Sub) and sent all messages and have received confirmation that all messages were received the client that pushed messages out still displays a huge memory footprint, apparently there are still huge amounts of messages in the Push socket’s queue. Why is that and what can I do to fix that?

Here is my code:

ROUTER:

class Program
{
    static void Main(string[] args)
    {
        using (var context = new Context(1))
        {
            using (Socket socketIn = context.Socket(SocketType.PULL), socketOut = context.Socket(SocketType.XPUB))
            {
                socketIn.HWM = 10000;
                socketOut.Bind("tcp://*:5560"); //forwards on this port
                socketIn.Bind("tcp://*:5559"); //listens on this port

                Console.WriteLine("Router started and running...");

                while (true)
                {
                    //Receive Message
                    byte[] address = socketIn.Recv();
                    byte[] body = socketIn.Recv();

                    //Forward Message
                    socketOut.SendMore(address);
                    socketOut.Send(body);
                }
            }
        }
    }
}

CLIENT1:

class Program
{
    static void Main(string[] args)
    {
        using (var context = new Context(1))
        {
            using (Socket socketIn = context.Socket(SocketType.SUB), socketOut= context.Socket(SocketType.PUSH))
            {
                byte[] iAM = Encoding.Unicode.GetBytes("Client1");
                byte[] youAre = Encoding.Unicode.GetBytes("Client2");
                byte[] msgBody = new byte[16];

                socketOut.HWM = 10000;
                socketOut.Connect("tcp://localhost:5559");
                socketIn.Connect("tcp://localhost:5560");
                socketIn.Subscribe(iAM);

                Console.WriteLine("Press key to kick off Test Client1 Sending Routine");
                Console.ReadLine();

                for (int counter = 1; counter <= 10000000; counter++)
                {
                    //Send Message
                    socketOut.SendMore(youAre);
                    socketOut.Send(msgBody);
                }

                Console.WriteLine("Client1: Finished Sending");
                Console.ReadLine();
            }
        }
    }
}

CLIENT2:

class Program
{
    public static int msgCounter;

    static void Main(string[] args)
    {
        msgCounter = 0;

        using (var context = new Context(1))
        {
            using (Socket socketIn = context.Socket(SocketType.SUB), socketOut = context.Socket(SocketType.PUSH))
            {
                byte[] iAM = Encoding.Unicode.GetBytes("Client2");

                socketOut.Connect("tcp://localhost:5559");
                socketIn.Connect("tcp://localhost:5560");
                socketIn.Subscribe(iAM);

                Console.WriteLine("Client2: Started Listening");

                //Receive First Message
                byte[] address = socketIn.Recv();
                byte[] body = socketIn.Recv();
                msgCounter += 1;

                Console.WriteLine("Received first message");

                Stopwatch watch = new Stopwatch();
                watch.Start();

                while (msgCounter < 10000000)
                {
                    //Receive Message
                    address = socketIn.Recv();
                    body = socketIn.Recv();
                    msgCounter += 1;
                }

                watch.Stop();
                Console.WriteLine("Elapsed Time: " + watch.ElapsedMilliseconds + "ms");
                Console.ReadLine();
            }
        }
    }
}

Answer 1

I’m going to suggest that your architecture may be a bit off here.

1) If you need exactly one PUSH and exactly one PULL, remove the device from the middle. Devices are added to an architecture explicitly to mange multiple consumers so that you don’t have to update producers each time you add a node. When/If you do get to where you need multiple consumers and/or producers, you’re going to need a connection to each node on your device – that’s just how they work. In this case, it sounds as though the device is overly complicating your solution.

2) The idea of having the “route to” tag really boggles my mind. Probably the biggest reason to choose messaging over other integration options is to decouple your producers and consumers so that neither side has to know anything about the other (other than where to send the messages in the case of broker-less designs). Adding routing information directly to your logic breaks this.

As to the overhead, I’ve never experienced this. But then, I’ve never used the .Net driver for ZeroMQ before and so an uneducated guess would be to look at the .Net driver itself.