Question

I have searched and have found almost nothing, so I would really appreciate some help with my question.

I am writting a DirectShow source filter which uses libav to read and send downstream h264 packets from youtube’s FLV file. But I can’t find appropriate libav structure’s fields to implement correctly filter’s GetMediType() and FillBuffer(). Some libav fields is null. In consequence h264 decoder crashes in attempt to process received data.

Where am I wrong? In working with libav or with DirectShow interfaces? Maybe h264 requires additional processing when working with libav or I fill reference time incorrectly? Does someone have any links useful for writing DirectShow h264 source filter with libav?

Part of GetMediaType():

VIDEOINFOHEADER *pvi = (VIDEOINFOHEADER*) toMediaType->AllocFormatBuffer(sizeof(VIDEOINFOHEADER));
pvi->AvgTimePerFrame = UNITS_PER_SECOND / m_pFormatContext->streams[m_streamNo]->codec->sample_rate; //sample_rate is 0
pvi->dwBitRate       = m_pFormatContext->bit_rate;
pvi->rcSource        = videoRect;
pvi->rcTarget        = videoRect;

//Bitmap
pvi->bmiHeader.biSize     = sizeof(BITMAPINFOHEADER);
pvi->bmiHeader.biWidth    = videoRect.right;
pvi->bmiHeader.biHeight   = videoRect.bottom;
pvi->bmiHeader.biPlanes   = 1;
pvi->bmiHeader.biBitCount = m_pFormatContext->streams[m_streamNo]->codec->bits_per_raw_sample;//or should here be bits_per_coded_sample
pvi->bmiHeader.biCompression = FOURCC_H264;
pvi->bmiHeader.biSizeImage = GetBitmapSize(&pvi->bmiHeader);

Part of FillBuffer():

//Get buffer pointer
BYTE* pBuffer = NULL;
if (pSamp->GetPointer(&pBuffer) < 0)
   return S_FALSE;

//Get next packet
AVPacket* pPacket = m_mediaFile.getNextPacket();
if (pPacket->data == NULL)
   return S_FALSE;

//Check packet and buffer size
if (pSamp->GetSize() < pPacket->size)
   return S_FALSE;

//Copy from packet to sample buffer
memcpy(pBuffer, pPacket->data, pPacket->size);

//Set media sample time
REFERENCE_TIME start    = m_mediaFile.timeStampToReferenceTime(pPacket->pts);
REFERENCE_TIME duration = m_mediaFile.timeStampToReferenceTime(pPacket->duration);
REFERENCE_TIME end      = start + duration;
pSamp->SetTime(&start, &end);
pSamp->SetMediaTime(&start, &end);

P.S. I’ve debugged my filter with hax264 decoder and it crashes on call to libav deprecated function img_convert().

Answer 1

You have to fill the right fields with the right values.

The AM_MEDIA_TYPE should contain the right MEDIASUBTYPE for h264.

And these are plain wrong :

pvi->bmiHeader.biWidth = videoRect.right;

pvi->bmiHeader.biHeight = videoRect.bottom;

You should use a width/height which is independent of the rcSource/rcTarget, due to the them being indicators, and maybe completely zero if you take them from some other filter.

pvi->bmiHeader.biBitCount = m_pFormatContext->streams[m_streamNo]->codec->bits_per_raw_sample;//or should here be bits_per_coded_sample

This only makes sense if biWidth*biHeight*biBitCount/8 are the true size of the sample. I do not think so …

pvi->bmiHeader.biCompression = FOURCC_H264;

This must also be passed in the AM_MEDIA_TYPE in the subtype parameter.

pvi->bmiHeader.biSizeImage = GetBitmapSize(&pvi->bmiHeader);

This fails, because the fourcc is unknown to the function and the bitcount is plain wrong for this sample, due to not being a full frame.

You have to take a look at how the data stream is handled by the downstream h264 filter. This seems to be flawed.