7 Ways WebRTC Solves Ultra-Low Latency Streaming

How Do You Get Ultra-Low Latency with WebRTC?

As we’ve covered before, WebRTC (Web Real-time Communication) is currently the best way to get ultra-low latency in video streaming. We’ve also mentioned – quite a few times– that Red5 Pro’s WebRTC implementation results in sub 250 ms of latency. That feature enables fully interactive live-streaming, making real-time communication possible.

The question remains “How does WebRTC actually achieve such ultra-low latency?”. This post will provide a lower level examination of the features that make WebRTC ultra-low latency delivery so fast and efficient. For a more high-level overview please take a look at our post on WebRTC.

What is Ultra-low latency video streaming?

Before we get into the ways that WebRTC solves ultra-Low latency streaming, it’s worth discussing the definition of ultra-low latency. What actually is considered ultra-low latency in live video streaming? Over the last ten years, what was considered to be “ultra-low latency” really depended on who you asked, and getting an exact answer was often difficult. That said, the broadcast industry has more recently come to a consensus, categorizing “ultra-low latency” as anything ranging from one second to seven seconds, and sub-one second video streaming as “real-time latency”. Higher latency protocols like HLS (HTTP live streaming) and MPEG-DASH are the most widely used protocols for this category, and for good reason. They scale easily using traditional CDNs, support dynamic adaptive streaming out of the box, and cover lots of use cases where sub-second latency doesn’t matter.

On the other hand, WebRTC is particularly adept at minimizing video latency in a live stream, consistently hitting sub-second latency. The reason for this is that WebRTC was originally conceived as a protocol for point-to-point streaming video communication for use cases like video conferencing, where sub-second latency is critical. WebRTC has since been adopted by the wider broadcast industry for one-to-many streaming use cases like sports betting, live auctions, and many other interactive applications that require real-time video streaming. Since this post is about how WebRTC achieves low latency, we will focus on the real time video streaming category.

Without further ado, let’s jump into how and why WebRTC gets these incredible low latency streaming results and how it compares to other video streaming protocols.

UDP vs TCP

TCP and UDP are transport layer protocols used for sending bits of data—known as packets—over the Internet. In the case of ultra-low latency live streaming, we are sending video packets. Both protocols use the IP protocol meaning that a packet sent by either TCP or UDP will be sent to an IP address.

The similarities stop there, however. TCP (Transmission Control Protocol) is a connection orientated protocol with built-in error recovery and retransmission. All the back-and-forth communication reordering packets and ensuring their complete delivery introduces latency. TCP is what is used for high latency http-based streaming protocols like HLS and DASH, and the overhead of TCP is one of the reasons that these videos streaming protocols have difficulties achieving an ultra-low latency live stream.

WebRTC uses UDP (User Datagram Protocol) to deliver a faster stream of information by eliminating error-checking of every packet. Video packets are sent directly to the recipient without having to properly order and retransmit them. Rather than waiting for confirmation, the sender keeps transmitting packets without error recovery. If packets are dropped, they are lost and will not be resent (except for NACK, but more on that later). Losing all this overhead means the devices can communicate more quickly.

Rather than containing all the sequencing data like with TCP, UDP depends upon the application-level RTP protocol to properly order the packets and know which ones to drop if they don’t fit the current timeframe. More on RTP below. Again, WebRTC was designed to use the more efficient UDP protocol for live streaming to meet the needs of two-way communication in video conferencing, where low latency delivery of the video was that most important aspect, and buffering for video quality wasn’t an option. Low latency video streaming with WebRTC is all about efficiency, and TCP simply wouldn’t work effectively for that use case.

RTP Efficiency

WebRTC uses the streaming protocol RTP to transmit video over the Internet and other IP networks.

RTP sends video and audio data in small chunks. Each chunk of data is preceded by an RTP header; the RTP header and data are, in turn, contained in a UDP packet. The data is organized as a sequence of packets with a small size suitable for transmission between the servers and clients. RTP streams carry the actual media payload encoded by an audio or video codec. The header is used to adapt to varying network conditions such as a single participant joining from a low bandwidth connection. When the application plays one packet, the following packets may already be in the stage of decompression or demultiplexing.

The Internet, like other packet networks, occasionally loses and reorders packets and delays them by variable amounts of time. To cope with these impairments, the RTP header contains timing information and a sequence number that allows the receivers to reconstruct the timing produced by the source. This timing reconstruction is performed separately for each source of RTP packets in the conference to adapt to real-time conditions.

Adjustments are made through a mixer that resynchronizes incoming audio packets to reconstruct the constant small chunk spacing generated by the sender. It then mixes those reconstructed audio streams into a single stream, translates the encoding to the corresponding quality setting, and forwards the packet stream. Those packets might be unicast to a single recipient or multicast on a different address to multiple recipients. As the internet doesn’t support multicast, WebRTC leverages a unicast version of RTP.

By consolidating essential information, RTP streamlines the process of media delivery. As with the UDP protocol by itself, when RTP is layered on top of UDP it also has less overhead which makes it much faster than other streaming solutions like HLS or DASH including their low latency variations. More than just a lightweight protocol, RTP also reduces latency by using a push method to send out the stream. We shall discuss that in more detail in the next section.

For a very detailed explanation of how RTP works please refer to the Internet Standard and this article.

Push vs. Pull

In order to deliver a stream, there must be a connection between the broadcaster and subscriber. Beyond that, the broadcaster must know what it needs to actually send to the subscriber. There are two different methods for doing that: push and pull.

HTTP protocols, like HLS and MPEG DASH, employ a pull approach where the client continually requests segments of video. This means that there is a constant stream of requests flowing back and forth between the broadcaster and subscribing client. It’s basically like trying to send an email line by line, rather than all at the same time. This constant polling creates a large amount of overhead, making for an inefficient process. It should be noted that there are advantages to this approach, such as the ability to create a stateless system; CDNs use this feature of pull to be able to distribute chunks of video across multiple server (edge) nodes. However, even though it makes scaling streams to large audiences easier, this technique significantly increases latency.

Alternatively, RTP pushes a continuous stream of data to the client. Without waiting for confirmation, RTP just sends the video straight through. RTP knows that line 2 comes after line 1, so it doesn’t waste any time asking what data packet needs to be sent next. This is a much more efficient process than the call and response method of pulling.

WebRTC is Natively Supported in the Browser

WebRTC works natively in all web browsers. All the encoding and decoding is performed directly in native code, as opposed to JavaScript making for an efficient process.

One approach to ultra-low latency streaming is to combine browser technologies such as MSE (Media Source Extensions) and WebSockets. In this setup, the solution relies on JavaScript to pull all the data from the video stream. While MSE and WebSockets are both individually written in native code, the way a developer needs to make them work together requires writing JavaScript. As JavaScript is an interpreted language, it means it’s just not as efficient as native code. In the case of the MSE WebSocket approach, JavaScript is used to extrapolate the information required to correctly transport the stream. Since JavaScript is less efficient, this makes for a slower process and increased latency, not to mention that WebSockets are also TCP based.

Another recent approach to low latency video streaming that is gaining traction is the use of Google’s UDP based protocol QUIC to transport video. However, the current version of leveraging QUIC for video streaming also involves the use of JavaScript APIs to decode the video, adding extra overhead and, thus, latency to the solution. These APIs, WebTransport and WebCodec specifically, are not universally supported across all browsers, with Safari on iOS being a major holdout today. Still, we think QUIC has advantages for uni-directional live streams, and is something the Red5 team is actively monitoring.

Web Standards Based

While not directly contributing to WebRTC’s ability as a streaming codec to maintain ultra-low latency from a technical standpoint, one of its greatest advantages lies in its foundation as a web standard. As a protocol that is standardized and maintained by the World Wide Web Consortium (W3C) and the Internet Engineering Task Force (IETF), WebRTC offers a level of reliability, interoperability, and continuous improvement that is critical for developers and businesses alike.

As a live streaming web standard, WebRTC’s universal compatibility, continuous improvement, robust security, and a developer-friendly environment make it uniquely positioned to provide ultra-low latency streaming. Its design and ongoing enhancements are geared towards minimizing delays, ensuring synchronous communication, and delivering an interactive user experience that is as close to real time as possible.

WebRTC’s status as a web standard is not just a badge of reliability; it’s a core feature that enables it to deliver the ultra-low latency streaming experiences demanded by today’s real-time communication applications. Whether it’s for live auctions, interactive education, or live sports broadcasting, WebRTC’s standardization is a key enabler of its exceptional performance and broad adoption.

Where do we go from here?

In summary, WebRTC stands out as the premier choice for ultra-low latency live streaming, offering a real-time communication experience that is unparalleled by traditional video streaming protocols like HLS and DASH. This post has illuminated the technical intricacies of WebRTC, from its reliance on UDP over TCP to enhance speed and reduce latency, to the use of RTP for efficient video and audio data transmission. The protocol’s push methodology further streamlines the streaming process, eliminating the overhead associated with the pull methods of HLS and MPEG-DASH.

WebRTC is getting stronger day by day. New web standards like WHIP and WHEP have recently created a single standardized approach for signaling. WHIP, in particular, is now opening up new hardware encoders like Osprey’s Talon 4k, which had previously relied on other protocols for ingest. Software-based WHIP client implementations like OBS are being introduced at a rapid pace, and the overall WebRTC live streaming ecosystem is growing quickly.

For those seeking to deploy live streams, especially in scenarios demanding interactivity, low latency, good video quality, and scale, such as live auctions, real-time collaboration, or live sports broadcasting, Red5 Pro’s WebRTC implementation presents a robust solution. Its sub-250 ms latency capability meets and sets the industry standard for “ultra-low latency” live video streaming.

If you’re aiming to leverage the power of WebRTC for your live streaming platform and want to ensure your streams are delivered with the utmost efficiency and minimal latency, we are ready to help. Whether your focus is on large-scale broadcasting, real-time engagement, or any other application where latency is critical, reach out to us. Let’s discuss how Red5 Pro’s WebRTC implementation can transform your live video streaming experience, ensuring your content is not just seen, but interactively experienced in real time. Reach out to us to discuss how we can support your streaming goals and take your platform to the next level of live video communication. Send us an email at info@red5.net or schedule a call.