I’m researching the ability to send custom timestamped data events over RTP. An example might be a sequence of chat messages that should remain synchronized with whatever audio/video is being streamed. These messages have no intrinsic audio or video interpretation; it would be up to the client software to do something appropriate (add them to a chat log, etc).
I found some evidence that people accomplish this with a custom RTP codec. I also saw some talk of custom RTP payloads. Any light that can be shed here would be appreciated.
I would also be interested in hearing about possible implementations outside of RTP.
For transport of custom data per RTP it is probably best to use a custom unassigned payload type (see list at http://www.iana.org/assignments/rtp-parameters/rtp-parameters.xml). A more flexible approach would use a dynamic payload type assignment (ref, RFC 3551).
The sending side would set up the RTP header (ref. https://www.rfc-editor.org/rfc/rfc3550#section-5.1) with this payload type and the time stamp from the real time media frame you want to be in sync with.
On the receiving end you would dispatch handling of the RTP data based on the payload type in header of the received RTP packet. The handling should probably allow for little bit of latency between arrival of the media and the custom packet and then (dis)play both together …
If you are working in Java, you can probably build your applciation based on the architecture and abstractions provided by the JMF (http://www.oracle.com/technetwork/java/javase/tech/index-jsp-140239.html).