The basic idea is to make the muxer copy bufs from HB's fifos to internal per-track queues as quickly as possible. Since all the HB fifos are always emptied whenever muxer runs, there should be no possibility of deadlock. (To guarantee that the HB fifos can always be emptied, the internal queues expand as needed.)
Since we now have all the different media streams buffered in the muxer, we can output the streams interleaved in whatever size chunks we want. I chose 4 times the video frame rate, 100-125ms, since it looks like that's what Apple uses for HD (they seem to use 200-250ms for SD or maybe they changed defaults & HD is done with a newer encoder) and it makes sense in terms of startup latency & player internal buffer requirements.
The internal queues are output as soon as all of them have enough data to make up a chunk so the internal queue sizes are kept as small as they can be consistent with the temporal decoupling between the HB video & audio fifos (i.e., you have to hang onto audio frames until the video frames for the same time range show up & that can take a *long* time since video encoding is about 100 times slower than audio). I've temporarily changed the log level of the muxer track completion msg from 2 to 0 and added the internal fifo size to it so we can get an idea of the audio / video mismatch & maybe use it to retune (lower) the HB fifo sizes:
Code: Select all
mux: track 0, 2077 frames, 10861850 bytes, 1001.63 kbps, fifo 32
mux: track 1, 4049 frames, 1726678 bytes, 159.23 kbps, fifo 256
mux: track 2, 2699 frames, 4836608 bytes, 446.01 kbps, fifo 256
Anyway, it would be nice if people could give this a try & report any problems. Thanks.