Question

I am struggeling with the Kinect for Windows SDK to create an application for conducting (with C#).

Basically I need to track one hand (usually the right one) of a conductor and recognize his speed in directing (BPM) to send this value to another application via MIDI.

What I started with is the SkeletonFramesReadyEvent adding the JointType.HandRight with a DateTime.Now.Ticks timestamp to a history List which is updated and removes the first entry. I keep the history of 60 frames (2 seconds).

I calculate the BPM by searching for the last low and high of the Joint.Position.Y and then calculate the difference and divide bpm = 60*ticksPerSecond/diff. However the result is wrong. Is there another way of doing this? What am I missing?

This is what I am using so far:

public int DetectBPM(JointType type)
{
    // we have not history yet
    if (!HasHistory()) return 0;

    // only calculate every second
    var detectTime = DateTime.Now.Second;
    if (_lastBPM != 0 && _lastBPMDectect == detectTime) return _lastBPM;

    // search last high/low boundaries
    var index = (int) type;
    var list = History[index];
    var i = list.Count - 1;

    var lastHigh = list[i];
    var lastLow = list[i];

    // shift to last peak first
    while (i > 0 && list[i].Joint.Position.Y >= list[i - 1].Joint.Position.Y) i--;

    // find last low
    while (i >= 0 && lastLow.Joint.Position.Y >= list[i].Joint.Position.Y) lastLow = list[i--];

    // find last high
    while (i >= 0 && lastHigh.Joint.Position.Y <= list[i].Joint.Position.Y) lastHigh = list[i--];

    var ticks = lastLow.Timestamp - lastHigh.Timestamp;
    var elapsedTime = new TimeSpan(ticks);

    var bpm = (int) (60000/elapsedTime.TotalMilliseconds);

    Console.WriteLine("DEBUG: BPM = " + _lastBPM + ", elapsedMS: " + elapsedTime.TotalMilliseconds);

    _lastBPMDectect = detectTime;
    _lastBPM = bpm;

    return _lastBPM;
}

Answer 1

I figured out how to do it. I was missing a point and calculated the BPM between a peak and a low position of the hand which is wrong. I have to calucalte the time difference between the last two low points to get the correct result.

The correct way to go is, find the stating point which is the last peak. From there move to the last low, this is the first point to calculate the difference from. The move to the next peak and go down to the next low again which is the second point to calculate the difference from.

The principle is shown in the figure below

And that results in a nicely BPM calculated like this:

var ticks = Math.Abs(firstLow.Ticks - secondLow.Ticks);
var elapsedTime = new TimeSpan(ticks);

var bpm = (int) (60000/elapsedTime.TotalMilliseconds);

Thanks for participating anyway.