Question

I found some callback signature such as data_completion_t, string_completion_t. But I wonder how zookeeper call those functions in asynchronous APIs.

Does it use a single thread to receive response from zookeeper? So I have to add mutex to protect user_data in callback. Or does it check callbacks every time another asyn API has been called?

Answer 1

I tested the C mt library, and found that zookeeper created another 2 threads for IO and completion, the completion function is called in the completion thread, so that those functions should be called sequentially.

2012-10-09 15:35:36,904:60028(0x7fff7d0f7180):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=127.0.0.1:3000 sessionTimeout=5000 watcher=0x0 sessionId=0 sessionPasswd=<null> context=0x0 flags=0
2012-10-09 15:35:36,904:60028(0x7fff7d0f7180):ZOO_DEBUG@start_threads@221: starting threads...
2012-10-09 15:35:36,905:60028(0x10af45000):ZOO_DEBUG@do_completion@459: started completion thread
2012-10-09 15:35:36,905:60028(0x10aec2000):ZOO_DEBUG@do_io@367: started IO thread
2012-10-09 15:35:36,905:60028(0x7fff7d0f7180):ZOO_DEBUG@zoo_aset@2700: Sending request xid=0x5073d3c9 for path [/mm/mmidc] to 127.0.0.1:3000
2012-10-09 15:35:36,905:60028(0x10aec2000):ZOO_INFO@check_events@1703: initiated connection to server [127.0.0.1:3000]
2012-10-09 15:35:36,909:60028(0x10aec2000):ZOO_INFO@check_events@1750: session establishment complete on server [127.0.0.1:3000], sessionId=0x13a43632d85000f, negotiated timeout=5000
2012-10-09 15:35:36,909:60028(0x10aec2000):ZOO_DEBUG@check_events@1756: Calling a watcher for a ZOO_SESSION_EVENT and the state=ZOO_CONNECTED_STATE
2012-10-09 15:35:36,909:60028(0x10af45000):ZOO_DEBUG@process_completions@2107: Calling a watcher for node [], type = -1 event=ZOO_SESSION_EVENT
2012-10-09 15:35:36,910:60028(0x10aec2000):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=1 xid=0x5073d3c9 rc=-101
zoo_set2: no node

the very simple code for the test, the zookeeper server is a standalone listening localhost:3000

  int main()
  {

    zoo_set_debug_level(ZOO_LOG_LEVEL_DEBUG);

    const char* host = "127.0.0.1:3000";

    zhandle_t *zh;
    clientid_t myid;
    zh = zookeeper_init(host, NULL, 5000, &myid, NULL, 0);

    struct Stat stat;
    const char* line = "/test";
    const char* ptr = "hello, world";
    int ret = zoo_set2(zh, line, ptr, strlen(ptr), -1, &stat);
    printf("zoo_set2: %s\n", zerror(ret));

  }

Actually, these are explained in the document. http://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html

C Binding


The C binding has a single-threaded and multi-threaded library. The multi-threaded library is easiest to use and is most similar to the Java API. This library will create an IO thread and an event dispatch thread for handling connection maintenance and callbacks. The single-threaded library allows ZooKeeper to be used in event driven applications by exposing the event loop used in the multi-threaded library.