We explored latency in H.264 transport for UAV/UAS and based on our analysis, much of the capture to display latency apparently comes from H.264 P and B frame reordering in a GoP, which requires the encoder to set a large PTS/DTS and the decoder to buffer at least one full GoP plus more for bandwidth delay. This could be almost a second of video delay for a GoP of 12 frames or more and say satellite relay of the digital telemetry link (compared to line of sight). This seems too high for tele-operation of a UAV/UAS for many situations. So, I was just curious what others are doing with H.264 when low latency is required or desired.
Seems like in fact the best answer is I & P frames only per "The H.264 Advanced Video Compression Standard, Second Edition", but Iain E. Richardson, based on comments in 6.4.7.1 on page 169 - "The prediction structure shown ...uses only I and P slices. It is compatible with the Baseline Profile or Constrained Baseline Profile of H.264, which do not allow B slices, and would be suitable for an application requiring low delay and/or minimal storage memory at the decoder." -- this is what I thought, but just wanted to confirm - this seems to confirm well. Although we found there still seems to be more delay than required for many encoders which set PTS/DTS higher than needed - does anyone else have this same experience?
We are using the x264 video encoder for video-based teleoperation of road vehicles. We managed to configure the encoder such that it doesn't produce any additional delay. Therefore we use the “baseline profile”, which like you said doesn't allow b-slices. Additionally we use the "verfast" preset in conjunction with the "zerolatency" tuning option. This tuning option disables all features which would result in additional encoding delays, such as rate control lookahead, multiple threads or B-frames.
To equally distribute the data rate to several frames we also use the "intra refresh" feature provided by the x264 encoder. This only encodes one slice per frame as an I-slice instead of a full I-frame.
With these settings you only get the delay resulting from en- and decoding one frame since the encoded data of one frame is directly returned by the encoder. The same goes for the decoder.
- Badges
- Science topic
- Similar topics
- Linguistics
- Composition Studies
- Publication