It would be fantastic if everyone could have a high level of connectivity; but that’s not always possible. We’re all familiar with the frustrating experience of watching people drop in and out of a Zoom conference — their video feed popping up from one side to another like a really boring game of whack-a-mole. Eventually their connection seems to stabilize only to freeze just as they are about to give their update. It’s a fragmented and exhausting process.

This freezing and disconnecting can stem from a poor network connection that is unable to handle the amount of data being sent through it. Or it could result from sending a high-quality feed, which requires a good amount of CPU power to process, to a less powerful smartphone. With a VOD solution you can add a buffer to allow everything to catch up. The added latency from the buffer doesn’t matter as much with recorded content. This approach, however, won’t work with live video, as real-life won’t pause for your internet connection to catch up.

One solution to address poor connectivity is to send a low-quality stream to everyone. However, this will downgrade the entire experience for all participants, including those with high-speed connections — far from a good solution. Although a 480p stream looks fine on a mobile device, watching the same stream on a full-sized desktop monitor is a different story.

The challenge is determining the best approach for each user — that is, delivering high-quality streams to users with outstanding bandwidth and processing power, while also accommodating those lacking in the speed and power department.

The best approach is to use adaptive bitrate streaming (ABR). With a typical ABR setup, a transcoder is used to create multiple streams at different bitrates and resolutions. This is usually done with media server software sending the multiple streams to a selective forwarding unit (SFU). The SFU will select the best stream for each client. Which stream the SFU selects often is based only on bandwidth; the best solutions, however, use other factors as well such as display size and CPU utilization.

Multiple renditions allow for playback on different bandwidths. The media server then sends the highest-quality stream possible for each viewer’s device and connection speed. And this is all done in the most efficient manner possible.

How Does ABR with WebRTC work?

Red5 Pro has implemented ABR with WebRTC on the server-side (unlike simulcast, which implements a similar method on the client-side). It does this by creating multiple resolutions so that the server can automatically select the best quality for the client’s current network conditions it is experiencing. ABR allows for dynamic upgrading and downgrading of the stream quality based on network conditions. That way each user can enjoy the smoothest possible live-streaming experience without having to think about it.

The process involves publishing a single high-quality stream to a transcoder node which then generates multiple variants at lower quality with configurable bitrate and resolution settings. Many applications will use three variants — high, medium, low — although any number can be generated. These variants are then streamed to an origin that re-streams to the edge server(s). As with all of Red5 Pro’s setups, the entire process is orchestrated by a stream manager.

Publishing a Stream

When publishing with WebRTC it is necessary to use the Stream Manager SSL proxy. The proxy will transfer the publish request to the transcoder provided. This is required because browsers demand that WebRTC broadcasts are served over SSL. Moreover, in an autoscale environment in which Red5 Pro instances are spun up and down dynamically based on load, it can be unfeasible to have certs on each of them.

Specifically, the act of publishing has four steps (fig. 1):

1. A provision is provided to the stream manager. It specifies the different variants of the stream that will be required. Then, the Stream Manager API is used to request a transcoder endpoint.
2. The stream manager selects a suitable origin and transcoder. The origin is provisioned to expect multiple versions of the same stream while the transcoder is provisioned with the details of the different stream variants that it needs to create.
3. The stream manager returns a JSON object with the details of the transcoder endpoint that should be used to broadcast.
4. The broadcaster can then start to publish a single stream to the transcoder.

Subscribing to a Stream

Subscribing with WebRTC involves using the stream manager proxy, which will transfer the subscribe request to the edge provided. This is necessary because browsers require that WebRTC uses SSL connections. The proxy avoids having to deploy a cert on every edge which would be unfeasible when using an autoscaling environment, where Red5 Pro instances are spun up dynamically based on load.

When a client wants to subscribe to a stream, it will have to request an edge from the stream manager. The request has to specify the stream name of the variant to subscribe to. Normally, it is best to start with a variant that is in mid-range and then let the edge make any adjustments if necessary. The edges use an RTCP message called Receiver Estimated Maximum Bitrate (REMB) to provide a bandwidth estimation. That estimate will then be used to determine what variant to serve to each subscriber based on the conditions of the network. It will continue to monitor the bandwidth and make adjustments as needed.

Once the stream manager has established a connection to the correct edge instance and the edge has selected the correct variant, the client can start watching the stream.

WebRTC’s Built-in ABR Support

Those familiar with WebRTC might know that WebRTC already supports ABR. So, why did we add all these extra adjustments?

The WebRTC protocol was designed for peer-to-peer browser connections. As such, in a server-to-browser model, it has built-in ABR on the subscriber side only. In a one-to-one situation (fig. 2), this works fine as the publisher can just change resolutions to adjust to the single subscriber’s need.

However, when multiple subscribers are involved, things are not as simple. If the stream isn’t transcoded to multiple qualities from the beginning, then a bad network on a single subscriber would request the publisher to switch to a lower-quality stream, forcing everyone to watch a low-quality stream.

Thus we implemented a transcoder to create those variants. The REMB message sent to and from the client ensures that the appropriate video is consistently delivered to each specific client without affecting other clients connected to the same stream.

ABR is an essential component of creating an effective live streaming experience. It allows you to adjust to a full range of devices and connectivity speeds so that each and every user can get the best possible quality no matter their streaming conditions. Importantly, the Red5 Pro configuration of ABR with WebRTC prevents the stream from being forced to cater to the lowest denominator. So when your coworker tries to join the meeting from their vacation home in the middle of who-knows-where, they can still participate in the call.

Interested in finding out more about creating the best live streaming experience? Send a message to info@red5.net or schedule a call. We’d love to show you all that we can do!