Question

I have several videos that I am trying to process using OpenCV and Qt 4.7.4 on Mac OS 10.6.8 (Snow Leopard). If I create a cv::VideoCapture object and then query for the frame rate related to such video, what I get back is the TBR and not FPS.

For instance if use ffprobe Video1.mp4 what I get is:

>> ffprobe Video1.mp4      
ffprobe version 0.7.8, Copyright (c) 2007-2011 the FFmpeg developers
built on Nov 24 2011 14:31:00 with gcc 4.2.1 (Apple Inc. build 5666) (dot 3)
configuration: --prefix=/opt/local --enable-gpl --enable-postproc --enable-swscale --   
enable-avfilter --enable-libmp3lame --enable-libvorbis --enable-libtheora --enable-
libdirac --enable-libschroedinger --enable-libopenjpeg --enable-libxvid --enable-libx264 
--enable-libvpx --enable-libspeex --mandir=/opt/local/share/man --enable-shared --
enable-pthreads --cc=/usr/bin/gcc-4.2 --arch=x86_64 --enable-yasm
libavutil    50. 43. 0 / 50. 43. 0
libavcodec   52.123. 0 / 52.123. 0
libavformat  52.111. 0 / 52.111. 0
libavdevice  52.  5. 0 / 52.  5. 0
libavfilter   1. 80. 0 /  1. 80. 0
libswscale    0. 14. 1 /  0. 14. 1
libpostproc  51.  2. 0 / 51.  2. 0

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Video1.mp4':
Metadata:
major_brand     : isom
minor_version   : 0
compatible_brands: mp41avc1qt  
creation_time   : 2012-01-09 23:09:43
encoder         : vlc 1.1.3 stream output
encoder-eng     : vlc 1.1.3 stream output
Duration: 00:10:10.22, start: 0.000000, bitrate: 800 kb/s
Stream #0.0(eng): Video: h264 (Baseline), yuvj420p, 704x480 [PAR 10:11 DAR 4:3], 798 
kb/s, 27.71 fps, 1001 tbr, 1001 tbn, 2002 tbc
Metadata:
  creation_time   : 2012-01-09 23:09:43

Which correctly reports FPS = 27.71 and TBR = 1001. Nevertheless if I use the following OpenCV code to query for the FPS:

QString filename = QFileDialog::getOpenFileName(this,
                                        "Open Video",
                                        "Video Files (*.mp4, *.mpg)");

capture.release();
capture.open(filename.toAscii().data());

if (!capture.isOpened()){
    qDebug() <<"Error when opening the video!";
    return;
}


qDebug() << "Frame Rate:" << capture.get(CV_CAP_PROP_FPS);
qDebug() << "Num of Frames:" << capture.get(CV_CAP_PROP_FRAME_COUNT);
qDebug() << "OpenCV Version" << CV_VERSION;

The output I get is:

Frame Rate: 1001 
Num of Frames: 610832 
OpenCV Version 2.3.1 

Which reports the TBR instead of the FPS. This behavior is consistent when I try to open different videos.

I checked OpenCV’s bug tracker and I also found this stack overflow question to be a similar but not quite the same problem, so I am at a loss to what to do next. Any hint or idea is most welcome since I’ve tried lots of things and seem to be getting nowhere.

Answer 1

I imagine they chose to report TBR because that is ffmpeg’s best guess as to what the framerate actually is. On many containers, the fps field (more specifically AVStream.avg_frame_rate) is not available, so it can’t really be relied upon.

Here are the comments from the ffmpeg source on the tbr, tbc, and tbn fields:

TBR (ffmpeg’s best guess):

struct AVStream {
...
/**
 * Real base framerate of the stream.
 * This is the lowest framerate with which all timestamps can be
 * represented accurately (it is the least common multiple of all
 * framerates in the stream). Note, this value is just a guess!
 * For example, if the time base is 1/90000 and all frames have either
 * approximately 3600 or 1800 timer ticks, then r_frame_rate will be 50/1.
 */
AVRational r_frame_rate;

TBC (the codec timebase):

struct AVCodecContext {
...
/**
 * This is the fundamental unit of time (in seconds) in terms
 * of which frame timestamps are represented. For fixed-fps content,
 * time base should be 1/framerate and timestamp increments should be 1.
 * decoding: set by libavformat
 * encoding: set by libavformat in av_write_header
 */
AVRational time_base;

TBN (the stream (container) timebase):

struct AVStream {
...
/**
 * This is the fundamental unit of time (in seconds) in terms
 * of which frame timestamps are represented. For fixed-fps content,
 * timebase should be 1/framerate and timestamp increments should be
 * identically 1.
 * - encoding: MUST be set by user.
 * - decoding: Set by libavcodec.
 */
 AVRational time_base;

Hopefully, that explains why TBR is reported instead of FPS. It appears ffmpeg is having difficulty determining the time base for your video stream, and the container (e.g., AVI, MP4, DivX, XviD, etc…) supplies the framerate, so ffmpeg displays it even though it can’t determine it through analysis. Is it possible to re-encode the video properly?