Question

I am writing a Kinect WPF application in C# to send gesture commands to aLinux C++ application.

My problem is when the Linux app’s framerate drops below the Kinect’s (25fps), it begins to register commands incredibly slow. A rotation would occur 5 or so seconds after the information was sent. From this I feel as if its emptying some queue of packets ever so slowly as they get backed up.

The framerate drops due to the increased model complexity being rendered on the Linux side.

At each rendering frame, I check if any packets were received for about 1ms through a select() and recvfrom(), then I parse it. Followed by updating the scenegraph for the world.
On the Kinect side, running at a solid 25fps is sending packets of gesture data as they are recognized.

Why is there a severe delay as the Linux FPS < Kinect FPS? It appears to be executing all the commands, just at a significant delay for a real time application.

Also, what are possibly solutions to alleviate this delay?

I think the delay is caused by the Linux’s inability to process all of the packets being sent in, so the packets get queued. However, I’m not sure why there is still a few seconds of no updates despite it technically reading 1 packet per frame.

Here is acode snippet.

(executes once for each frame)

  tv.tv_sec = 0;
  tv.tv_usec = 1*1000

      //BLOCKING 1ms  
  retval = select(sfd+1, &fds, NULL, NULL, &tv);

  if (retval < 0) {
     perror("select");
  } else if (retval) {
     int n = recvfrom(sfd, recvbuf, 1024, 0, (struct sockaddr *)&caddr, &len);
     parse_command(recvbuf);
  }

//After this it begins to update/transform the world/scene

Thanks!

Answer 1

What are possibly solutions to alleviate this delay?

A very simple method would be to use acknowledgement packets. Send a bunch of messages (up to a chosen threshold, for example, 1 second’s worth). As long as the acknowledgement packets are received, you will continue to send data packets about what is happening on the Kinect. If you do not receive any acknowledgements (so your Linux app is running a little behind), then you will drop data until it catches up (and you will know because of the ACK packet).

Why is there a severe delay as the Linux FPS < Kinect FPS? It appears
to be executing all the commands, just at a significant delay for a
real time application.

This has to do with the way your code and how you’re executing it. Without any information about your parse_command() function, we will not know what. Yes, you’re executing all the commands, but you are taking too long to execute them. At 25 FPS, you have 40ms before you start seeing lag. Within that time, you need to do all your work parsing, rendering and your commands.

I think the delay is caused by the Linux’s inability to process all of
the packets being sent in, so the packets get queued. However, I’m not
sure why there is still a few seconds of no updates despite it
technically reading 1 packet per frame.

Please do not make assumptions about software (or anything in general) without doing some research and having some conclusive evidence. Linux is perfectly capable of receiving large amounts of data through a network. In fact, it is very performant and can receive gigabits of data at a time. The issue is with your client side code, not the kernel.