Question

I am at a point in my Erlang development where I need to create a C-Node (see link for C-Node docs). The basic implementation is simple enough, however, there is a huge hole in the doc.

The code implements a single threaded client and server. Ignoring the client for the moment… The ‘c’ code that implements the server is single threaded and can only connect to one erlang client at a time.

1. Launch EPMD (‘epmd -daemons’)
2. Launch the server application (‘cserver 1234’)
3. Launch the erlang client application (‘erl -sname e1 -setcookie secretcookie’) [in a different window from #2]
4. execute a server command (‘complex3:foo(3).’) from the erlang shell in #3

Now that the server is running and that a current erlang shell has connected to the server try it again from another window.

1. open a new window.
2. launch an erlang client (‘erl -sname e2 -setcookie secretcookie’).
3. execute a new server command (‘complex3:foo(3).’).

Notice that the system seems hung… when it should have executed the command. The reason it is hung is because the other erlang node is connected and that there are no other threads listening for connections.

NOTE: there seems to be a bug in the connection handling. I added a timeout in the receive block and I caught some errant behavior but I did not get them all. Also, I was able to get the cserver to crash without warnings or errors if I forced the first erlang node to terminate after the indicated steps were performed.

So the question… What is the best way to implement a threaded C-Node? What is a reasonable number of connections?

Answer 1

The cnode implementation example in the cnode tutorial is not meant to handle more than one connected node, so the first symptom you’re experiencing is normal.

The erl_accept call is what accepts incoming connections.

if ((fd = erl_accept(listen, &conn)) == ERL_ERROR)
  erl_err_quit("erl_accept");
fprintf(stderr, "Connected to %s\n\r", conn.nodename);
while (loop) {
  got = erl_receive_msg(fd, buf, BUFSIZE, &emsg);

Note that, written this way, the cnode will accept only one connection and then pass the descriptor to the read/write loop. That’s why when the erlang node closes, the cnode ends with an error, since erl_receive_msg will fail because fd will point to a closed socket.

If you want to accept more than one inbound connection, you’ll have to loop accepting connections and implement a way to handle more than one file descriptor. You needn’t a multithread programme to do so, it would probably be easier (and maybe more efficient) to use the poll or select syscall if your OS supports them.

As for the optimum number of connections, I don’t think there is a rule for that, you’d need to benchmark your application if you want to support high concurrency in the cnode. But in that case it would probably be better to re-engineer the system so that erlang copes with the concurrency, alleviating the cnode from that.