Home/ Questions/Q 6771207
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-26T15:24:18+00:00

In previous version of my RTSP Source Filter , I was immediately packing the

  • 0

In previous version of my RTSP Source Filter, I was immediately packing the RTP packets in order I receive them. There was no buffer, sorting by sequence number, dropping frames with missing pieces etc. but result was ok since I made my tests on LAN. I decided to add an RTP buffer, sorting mechanism etc. Actually, core code was successful. But now I have some problems with sending the frames with correct timing.

I have a debugging mechanism. I’m able to store all the frames into seperate files like frame0.bin, frame1.bin etc. and I have a tool which is able to read these files and send it to h.264 decoder. When I playback these frames, result is perfect. I believe that proves my RTP and frame buffers work fine.

When I try to pass the frame to FillBuffer function as soon as a frame is created (actually when my frame queue has more than 1 frames), result is really crappy (I get the image but with delays and corrupted frames). It’s probably caused by passing frames to FillBuffer function immediatelly. My DoBufferProcessingLoop function looks like below (I removed most of the error checks from the code);

HRESULT RtspSourceFilterOutputPin::DoBufferProcessingLoop() {

    Command com;
    REFERENCE_TIME rtNow = 0L;
    REFERENCE_TIME rtAdvise = 0L;

    OnThreadStartPlay();

    do {
        while (!CheckRequest(&com)) {
            if(streamReader->frames.size() > 1) {
                IMediaSample *pSample;
                GetDeliveryBuffer(&pSample,NULL,NULL,FALSE);

                hr = FillBuffer(pSample);

                if (hr == S_OK) {
                    HRESULT result = Deliver(pSample);
                } else if (hr == S_FALSE) {
                    pSample->Release();
                    DeliverEndOfStream();
                    return S_OK;
                } else {
                    //error
                }
                pSample->Release();
            }
        }

        if (com == CMD_RUN || com == CMD_PAUSE) {
            com = GetRequest();
        } else if (com != CMD_STOP) {
            //error
        }
    } while (com != CMD_STOP);

    return S_OK;
}

Since streaming is realtime, I removed all timestamps (SetTime, SetMediaTime etc.) from FillBuffer function. There is nothing special with FillBuffer. It just pops a frame from frames queue and pass it to the decoder. And it works if a remove my buffer mechanism.

Finally, the quesiton is…

Should I wait for some time like “received time + buffer time” before sending a frame instead of passing it to FillBuffer as soon as its created? Or should I use SetTime in FillBuffer function? I already tried it and failed. I also tried to give start/stop time values with buffer time added but didn’t work any better.

Why is the result crappy when I pass the frame to the decoder but result is ok if I save these frames to binary files and playback from these files? What am I missing here?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    The fact that you’re buffering shouldn’t make any difference as long as your samples are time-stamped correctly. If you don’t time-stamp samples (i.e. call SetTime with NULL as in

    pSample->SetTime(NULL, NULL);

    the samples will be rendered by the renderers as quick as possible. If you do set the time stamps, the renderers will render the frame accordingly (wrt. the current stream time). Perhaps your buffering is causing samples to arrive late? In any case, setting the time stamps to NULL is a good way to test whether all the media is received and is in order.

    So to answer your question, yes you do need to set time stamps on the samples: real-time doesn’t mean you that you can remove the timestamps. Network jitter will cause the arrival times of samples to differ and the time stamps must be set correctly if you want to view the stream(s) correctly. You don’t need to wait any time before passing the samples to FillBuffer, as long as the time stamp is set, the renderer will take care of playing back the samples correctly.

    My approach, which has been working fairly well, is as follows:
    In the RTSP source filter perform the required buffering. Then, once the buffer has been filled, calculate the offset that will be used for each sample.
    You need to take into account the RTP timestamp of the first RTCP-synchronized sample received (ts_0) and the DirectShow StreamTime (st_0) at that point in time where you start playing back the samples. By the time you start passing samples through the pipeline, the stream time will likely not be zero anymore….

    The DirectShow timestamp ts_new of each sample will then be calculated as the 

    ts_new_x = ts_x - ts_0 + st_0 + 50ms

    The 50ms is useful to set the time stamp of the sample a little into the future. The reason for this is to avoid the samples arriving at the renderer late.
    The end time of the sample can be set as ts_new_x + 1. IIRC this gets ignored anyway, but don’t quote me on that.

    I would recommend reading all the sections in http://msdn.microsoft.com/en-us/library/windows/desktop/dd407202(v=vs.85).aspx a couple of times if you haven’t already. I found this to be extremely useful during the implementation.

    Also, feel free to have a look at the open source version of an RTSP source filter I wrote. It’s main purpose was to highlight some aspects of writing a live RTSP DirectShow source filter and supports PCM, AMR and MP3. I did add some minor H.264 support for it, but never got round to completing that part (IIRC it was able to play the live555 sample video streams).

    • 0