Question

I have a Silverlight application that uses an overridden AudioSink.OnSamples() to record sound, and MediaStreamSource.GetSampleAsync() to play sound.

For instance:

protected override void GetSampleAsync(MediaStreamType mediaStreamType)
{
    try
    {
        logger.LogSampleRequested();
        var memoryStream = AudioController == null ? new MemoryStream() : AudioController.GetNextAudioFrame();
        timestamp += AudioConstants.MillisecondsPerFrame * TimeSpan.TicksPerMillisecond;
        var sample = new MediaStreamSample(
            mediaStreamDescription,
            memoryStream,
            0,
            memoryStream.Length,
            timestamp, // (DateTime.Now - startTime).Ticks, // Testing shows that incrementing a long with a good-enough value is ~100x faster than calculating the ticks each time.
            emptySampleDict);
        ReportGetSampleCompleted(sample);
    }
    catch (Exception ex)
    {
        ClientLogger.LogDebugMessage(ex.ToString);
    }
}

Both of these methods should normally be called every 20 milliseconds, and on most machines, that’s exactly what happens. However, on some machines, they get called not every 20 ms, but closer to 22-24 ms. That’s troublesome, but with some appropriate buffering, the audio is still more-or-less usable. The bigger problem is that in certain scenarios, such as when the CPU is running close to its limit, the interval between calls rises to as much as 30-35 ms.

So:

(1) Has anyone else seen this?

(2) Does anyone have any suggested workarounds?

(3) Does anyone have any tips for troubleshooting this problem?

Answer 1

For what it’s worth, after much investigation, the basic solution to this problem is simply not to use as much CPU. In our case, this meant keeping track of the CPU utilization, and switching to a codec that didn’t use as much CPU (G711 vs. Speex) when the CPU starts consistently running at 80% or higher.