If I am streaming (I mean, simply broadcasting raw data over UDP sockets) a H.264 stream, how does the decoder decide at what FPS it should display the video?
I know that every once in a while it displays a frame that it receives, but how does it decide when the time is up for a frame and it is time to display the next one?
The playback framerate is determined by timecodes embedded in the transport stream, or in the video bitstream itself. Each frame is packaged with a display time (either relative or absolute) and this is used to decide how long each frame should remain on the screen.