Question

I seek to use the http live streaming standard with video. I’d like to eliminate any delay while a user is working with our app, but the current architecture requires fully encoding audio with any new or removed video clips.

Is there an incremental encoding approach to http live streaming so that I can

1. keep the audio track separate, but playback seamlessly with the video stream

2. allows .ts chunks to be independently encoded and streamed back to a user faster than re-encoding an entire video

References:
https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming

https://developer.apple.com/streaming/

Answer 1

You could re-encode the required segments fairly easily — there’s no need to have the entire stream encoded before playing it (otherwise live events would be impossible). You have to be careful with the timestamps in the TS packets if you want it to be truly seamless. But what might be easiest is to use EXT-X-DISCONTINUITY markers around the re-created portions.

As for audio, there’s no need to re-encode it. You should be able to just copy the encoded audio from one TS container to another. For example, if you’re using ffmpeg, you would use -acodec copy to take it from the original ts.