Live streaming is a complicated process involving a variety of different methods and formats. One of these components is the codec used for the encoding and decoding of the media file. The codec also defines the types of tools that the codec can use to conduct the streaming.

To greatly simplify the process; in order for a video to be streamed over the internet, it must first capture the audio and video using a microphone and camera. Then that raw data must be compressed (encoded) into a codec, broadcast over an internet connection (using a transport protocol), sent to some kind of server-side solution (typically a CDN or a cloud based cluster like Red5 Pro), and subsequently decompressed (decoded) for the subscriber to finally watch the video.

There are a few different codecs currently in use today including, VP8/9, AVC (h.264), HEVC (h.265), and AV1. In a previous post, we covered AV1, and we plan to cover VP8 and VPN9 in the near future, so this post will focus specifically on why AVC (h.264) is a more effective choice for low latency live streaming over setting up an HEVC (h.265) streaming server solution.

AVC vs. HEVC – What Is the Difference?

HEVC (High Efficiency Video Coding), also known as h.265, is the latest international standard for video compression. It is the successor to AVC (h.264) (first made available in 2003) which was preceded by MPEG-4. Developed by ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Video Coding Experts Group), the HEVC standard was first made available in 2013.

Logically, one might assume that since HEVC is the most recently developed codec it would be the most effective. The reality, however, is more complex. HEVC does offer advantages in terms of compression efficiency with an HEVC file taking up about half as much storage space as an equivalent ACV file. A smaller file size means it requires less storage space which, more importantly, corresponds to less bandwidth consumption when streamed. This advanced compression efficiency is due to how HEVC processes macroblocks. HEVC also features better variable-block-size segmentation, improved deblocking and motion compensation filters, sample adaptive offset filtering, and better motion vector prediction and precision. This page here from the x265 group has a great explanation of these terms and how it can improve HEVC’s efficiency.

With all these advantages, why isn’t HEVC the better codec for live streaming?

1) Higher Encoding Speeds

A big reason concerns latency. Even though HEVC ultimately produces a smaller video that can be sent faster since there is less data to send, all that extra compression (as compared to AVC) means that it will take much longer to perform the actual compression. Thus, it will make it longer to send the video. It may take less bandwidth to send but more time to do the actual compression which ultimately surpasses the time saved in stream delivery.

Although it might seem really obvious to those of us in the streaming industry, it’s worth explaining why it matters. Latency is important since any sort of delay between the broadcaster and subscriber will stifle an interactive live streaming experience. Any sort of live stream application needs low latency in order to have a natural flow. Sports and events broadcasts need to prevent spoilers or allow for crowd response in real-time, live auctions need to ensure bids are getting registered at the right time, and social platforms need to ensure smooth back and forth conversations.

Since HEVC has more overhead, that can lead to latency when compared to encoding h.264.

2) Limited Adoption

In order to work with the codec, there needs to be supported hardware or software encoders. HEVC suffered from a low adoption rate due in no small part to patent licensing. While AVC has only 1, HEVC has four patent pools related to it: HEVC Advance, MPEG LA, Velos Media, and Technicolor. This makes it more expensive which has discouraged more widespread adoption thus limiting it to specific hardware encoders and mobile chipsets. Additionally, in 2013 Cisco open-sourced its h.264 implementation and released it as a free binary download. That was a gigantic boost to the widespread implementation of AVC.

While there are HEVC software encoders available, the use of software encoders will create performance issues as it will consume more CPU. Live streaming is already a CPU intensive process so increasing the already high usage will be a problem. Accordingly, battery life will decrease as the latency potentially increases.

3) Not supported in Browsers

As opposed to AVC, HEVC is not supported by the majority of internet browsers. Only Edge, Internet Explorer and Safari support HEVC, and even then the device running the browser will still need to support HEVC hardware encoding. Even when HEVC is supported in browsers with the correct implementation, WebRTC tends to not work correctly.

With the widespread use of mobile devices and laptops for internet browsing, not being able to watch the video in internet browsers severely limits how users can access content. Supporting a wide variety of devices is the only way to ensure that a variety of users can watch content how they prefer to access it.

4) Reducing Bandwidth Is More Important for High Resolutions

Part of the advantage of higher compression rates and smaller file sizes is that it consumes less bandwidth meaning that users can still stream high-quality videos at lower internet speeds. The reality is that with a global average of 42.63 Mbps upload speed for fixed broadband connections, most places have internet speeds high enough to support even 4K streaming over AVC. Mobile devices have significantly lower average speeds at 10.93 Mbps, but even that is more than fast enough for 1080p.

This diagram from Boxcast shows that the average worldwide connection speeds are definitely able to handle the upload speed requirements at all tiers of resolutions.

For users in countries with slower internet speeds, adaptive bitrate support can adjust the bitrate to deliver the best experience and transcoding can split broadcasts into multiple qualities for the best experience depending upon the available bandwidth. When it comes to streaming to mobile devices with 2 or 3G connections, devices the size of your hand don’t even need 1080p resolutions to look good. 720 or even 480 will still display with good quality.

However, it must be acknowledged that smaller files will save companies money in regards to bandwidth costs. Smaller files mean not paying as much for more data streaming over CDN or cloud networks. While that is certainly nice, it is only at really high-resolution settings such as 4K that halving the data consumption makes a substantial difference.

Of course, saving money is certainly an important thing no matter what the scale. That brings us to our next point which will present the best of both worlds; better compression with the same performance.

5) LCEVC Circumvents the Whole Argument

LCEVC (Low Complexity Enhancement Video Coding) adds an extra processing layer to increase compression ratios in the 40% range with existing and future versions of MPEG or other codecs such as VP9 and AV1. As we covered in a previous article, LCEVC is a very promising piece of technology, as it essentially makes the current batch of protocols that have already been adopted in one form or another into more efficient protocols in and of themselves.

From where everything stands now, it looks like content providers will be able to use LCEVC-enabled software- or hardware-based encoders in conjunction with the Red5 Pro cross-cloud platform to facilitate real-time streaming with a new generation of processing-intensive video formats. Depending on which core codec is used, this applies not only to 4K and, eventually, 8K UHD, but also to formats devised for 360-degree viewing, virtual reality, and other innovations.

That’s because virtually any device can support a thin LCEVC client embedded in the service provider’s app player or downloaded independently to viewers’ devices. Furthermore, LCEVC supports plugin-in free browser support through the accompanying HTML5 JavaScript implementation. This means that widespread implementation should be fairly straightforward.

Nonetheless, AV1 is poised to replace both h.264 and h.265. The consortium behind it has all of the major players involved and it’s royalty-free. All that is holding AV1 back right now is the lack of real-time encoders. Once those become widely available, AV1 (especially when paired with LCEVC) will be the way forward.

Bottom Line: h.264 / AVC Works Better

Though HEVC certainly improves upon aspects of AVC, those improvements are not big enough to surmount the disadvantages. With higher latency due to longer encoding time, patent licensing concerns limiting adoption, the lack of browser support, and the real-world viewing resolution of most live streams HEVC doesn’t quite live up to expectations. Overall, an AVC streaming server will be much more effective than an HEVC streaming server. Even more important, LCEVC is poised to sidestep the entire codec issue making it all but null and void.

Think we missed something in our analysis? Let us know by sending an email to info@red5.net or schedule a call with us to find out more about Red5 Pro.