Video Latency in Live Streaming

A number of steps in the glass-to-glass journey affect video latency:

• Video encoding pipeline duration
• Ingest and packaging operations
• Network propagation and transport protocol
• Content delivery network (CDN)
• Segment length
• Player policies

Other methods exist, but the easiest way to measure end-to-end video latency is as follows:

1. Use a tablet running a clapperboard application
2. Film it with a camera connected to your video encoder
3. Publish your video stream to your origin
4. Deliver to your player through a CDN
5. Put the player alongside the clapperboard tablet
6. Take a picture of the two screens
7. Do the timecode subtraction, and you have your number
   

That OTT video latency lags behind broadcast TV and social media is not the only concern on content providers’ minds. Here are a few other issues to consider in achieving low latency:

Flash and Real-Time Messaging Protocol (RTMP): Flash-based applications using RTMP streaming used to work well for low latency, but with the deprecation of Flash and web browsers steadily reducing support for or blocking the plug-in, Content Delivery Networks (CDNs) have also begun deprecating RTMP – which already had limited scale on the delivery side – forcing content providers to take alternative action.

Scale and Reliability vs. Low Latency: One choice that solves scale problems: Switching to HTML5-friendly streaming technologies, such as HTTP Live Streaming (HLS), Dynamic Adaptive Streaming over HTTP (DASH or MPEG-DASH), and the Common Media Application Format (CMAF).

These streaming technologies distribute over HTTP, which means delivery is cacheable, and CDNs can deliver higher volumes with greater efficiency.

However, with both scale and reliability achieved, tens of seconds are added to end-to-end delivery times, tipping the scales against low latency.

Interactive Features: Other content providers may choose to develop personal broadcasting services with interactive features; however, a video signal delay is generally not acceptable for these use cases.

The reason: If video is delayed by up to 30 seconds from the time it is first captured on a camera to the time it is viewed, interactivity that requires real-time feedback is rendered impossible.

In addition, those who want to develop synchronized second screen, social watching, gaming, or gambling applications need to control video streaming latency on a fine-grained level.

Video latency can be viewed in three distinct categories, each defined by high and low boundaries.

However, the ways reduced video latency, low video latency and ultra-low video latency differ from broadcast video latency is worth knowing, if a little tricky to explain.

Six seconds of broadcast video latency is often the average in the field, which means that the sweet spot for OTT video latency is somewhere in the low range of the reduced video latency category, or in the high range of the low latency category. Getting close to five seconds maximizes the chances of competing with broadcast and social network feeds.

Moreover, depending on the OTT video encoder’s location in the content preparation workflow, the latency reduction objective increases as encoders are positioned further downstream in the chain.

At a high level, here are the main actions to conduct in order to get your video streaming solution into the low latency category:

• Measure video latency at every step in the workflow
• Optimize your video encoding pipeline
• Choose the right segment duration for your requirements
• Build the appropriate architecture
• Optimize (or replace) your video player(s)
  

The segment duration you choose has a mechanical effect on the video latency in almost every player. For example, you can achieve five-second latency with one-second duration segments. Choose two-second duration segments, and the result will be between seven and 10 seconds of video latency – unless you optimize the player’s settings.

The fundamental rule is to “right size” based on your requirements. So, if video latency of seven seconds or below is not critical, choose two-second segments.

If your player uses two-second segments, raise the GOP length from one to two seconds to increase the encoding quality at constant bitrate.

In addition, if you use HLS as the ingest format, you can reduce stress on the origin storage and packaging computation by using two-second segments when ingesting.

• Video latency for live streaming is not an insurmountable problem. You can minimize it, with some efforts
• Standard HLS and DASH technologies enable scalable low latency over HTTP
• Today’s video latency standard for live streaming: Less than 10 seconds
• A stable four-five seconds of video latency is possible now, if your business requires it
• The chunked CMAF ecosystem is maturing and will soon enable stable sub-four second video latency
  

Learn how to achieve predictable, stable low-latency video streaming using standard HLS or DASH protocols, and about the high-performance capabilities of cloud video origination and storage integrated with AWS in this AWS Elemental MediaStore demonstration.