Question

I run the following C program between two machines with 10GibE; the program reports 12Gib/s whereas nload reports a (more believable) 9.2Gib/s. Can anyone tell me what I’m doing wrong in the program?

.
.
#define BUFFSZ (4*1024)
char buffer[BUFFSZ];
.
.
  start = clock();

  while (1) {
    n = write(sockfd, buffer, BUFFSZ);
    if (n < 0)
       error("ERROR writing to socket");
    if (++blocks % (1024*1024) == 0)
    {
      blocks = 0;
      printf("32Gib at %6.2lf Gib/s\n", 32.0/(((double) (clock() - start)) / CLOCKS_PER_SEC));
      start = clock();
    }
  }

This is CentOs 6.0 on Linux 2.6.32; nload 0.7.3, gcc 4.4.4.

Answer 1

Firstly, clock() returns an estimate of the CPU time used by the program, not the wall-clock time – so your calculation indicates that you are transferring 12GiB per second of CPU time used. Instead, use clock_gettime() with the clock ID CLOCK_MONOTONIC to measure wall-clock time.

Secondly, after write() returns the data hasn’t necessarily been sent to the network yet – merely copied into the kernel buffers for sending. This will give you a higher reported transfer rate at the start of the connection.