9 Reasons Why LCEVC Circumvents the Codec Wars

For anyone wishing there was a way to meet bandwidth and latency challenges posed by 4K UHD and other emerging high bitrate video formats without having to wrestle with all the issues posed by next-gen codecs, here’s some good news: It looks like your wish has come true.

Earlier this year, the Moving Picture Experts Group (MPEG) took major steps toward final approval of Parts 1 and 2 to MPEG-5, known, respectively, as Essential Video Coding (EVC) and Low Complexity Enhancement Video Coding (LCEVC), and H.266, also known as Versatile Video Coding (VVC). All three standards are on track for final approval before year’s end, although there could be some slippage owing to uncertainties related to the pandemic and the upheaval at MPEG, which recently underwent reorganization under administration of ISO/IEC JTC 1 (Joint Technical Committee of the International Standards Organization and the International Electrotechnical Commission).

The only standard in this group that truly is a next-generation codec is H.266, which is meant to outperform the new, purportedly royalty-free AV1 codec backed by the Alliance for Open Media (AOMedia), whose members include Amazon, Apple, Facebook, Google, Microsoft, Cisco, Intel, IBM, Netflix and Samsung, among others. EVC is designed as a superior, lower-cost alternative to AVC (H.264) and HEVC (H.265) using elements of each, where 25-30% compression gains are achieved for AVC with a baseline royalty-free profile and for HEVC with a second profile that incurs royalty costs.

From a practical perspective, the standout in this codec stew is LCEVC, which isn’t exactly a new codec. Instead, LCEVC makes it possible to circumvent the codec wars by adding an extra processing layer to increase compression ratios in the 40% range with existing and future versions of MPEG or other codecs such as VP9 and AV1. And it does this while maintaining backward compatibility with the core codec, which means any device equipped to decode in the core format that isn’t running the LCEVC client software will be able to display the video at the level of resolution supported by the baseline encode without the enhancement.

Moreover, LCEVC actually reduces total encoding time by a significant amount compared to the time consumed by any given baseline codec to achieve a targeted level of resolution without LCEVC. This is a potential game changer, especially for live streaming.

The enhancement processing technology employed by LCEVC was developed by U.K. startup V-Nova and has gained commercial traction as a proprietary solution with deployments in India, Africa and other bandwidth-starved parts of the world. In making the technology available for MPEG standardization, V-Nova hasn’t announced licensing terms but has made clear costs associated with LCEVC will be minimal as compared to fees incurred with HEVC.

In a streaming world where AVC is the dominant codec, distributors who can implement a low-cost means of improving compression efficiency stand to realize major cost savings with reductions in the traffic loads they are running over transport links and through CDN tollgates. Huge costs savings can also be had with cloud based deployments like Red5 Pro where cloud providers can charge significant data transfer rates.

And when it comes to introducing UHD or other advanced video formats, there are major cost savings to be realized not only by avoiding HEVC royalty fees but also by eliminating the encoder and other expenses incurred with introducing a next-gen codec even when royalties aren’t as big a concern. In the case of LCEVC, the additional processing is accommodated via software running on commodity appliances with no need to replace existing encoders.

MPEG set daunting requirements for LCEVC, which many industry observers and vendors doubted could be met. But, remarkably, all benchmarks have been achieved, as evidenced by the progress of the standard toward finalization and by recently reported results from testing performed by independent parties.

First and foremost, when LCEVC is used to enhance a codec like AVC, MPEG says,“the compression efficiency for the aggregate stream is appreciably higher than that of the n-th generation MPEG codec used at full resolution and as close as possible to that of the (n+1)-th generation MPEG codec (e.g., HEVC) used at full resolution, at bandwidths and operating conditions relevant to mass market distribution.” In other words, you get HEVC-level compression using AVC encoding with no diminution in quality.

If all goes as indicated with initial testing, it looks like content providers will be able to use LCEVC-enabled software- or hardware-based encoders in conjunction with the Red5 Pro cross-cloud platform to facilitate real-time streaming with a new generation of processing-intensive video formats. Depending on which core codec is used, this applies not only to 4K and, eventually, 8K UHD, but also to formats devised for 3600 viewing, virtual reality and other innovations.

For example, live 4K UHD sports streams, typically requiring 10-bit encoding and a 60-fps frame rate, could be streamed using LCEVC with H.264. Rather than requiring an untenable 50-60 Mbps bitrate, these parameters could be supported at 30-36 Mbps, which has become a reasonable target amid ongoing bandwidth expansion in seasoned broadband markets.

Of course, a 4K sports stream delivered to native H.264 decoders would be transmitted at a much lower bitrate, resulting in a low-quality viewing experience. But this is likely to be of little consequence in situations where the content is streamed to authorized users in high-speed broadband environments.

That’s because virtually any device can support a thin LCEVC client embedded in the service provider’s app player or downloaded independently to viewers’ devices. And, eventually, there should be virtually no barriers to scaling LCEVC reach, because, as required by MPEG, the technology makes it possible via HTML5 JavaScript for web browsers to support LCEVC decoding without plug-ins or browser upgrades.

In cases where HEVC or a next-gen codec like AV1 or the new MPEG VVC is brought into play, the bitrates required for live streaming in 4K would be reduced by margins similar to those achieved with H.264, and it would be easier from a bandwidth perspective to support more bandwidth-intensive applications like VR. For example, with HEVC in the case of 4K, live sports video that would normally require transmissions in the 25-30 Mbps at the frame rates and bit-coding levels cited above could be streamed at 15-18 Mbps.

It’s also important to note that all these benefits apply when it comes to encoding for HD and SD.

For example, an H.264-encoded 1080p live sports stream that might normally require a top bitrate setting around 8 Mbps can be delivered at equivalent quality at 4.8 Mbps when the LCEVC processing layer is used, or about the same rate that would be used with an HEVC encode.

LCEVC also stands to create new opportunities for live and on-demand video distribution in low-bandwidth markets where people largely rely on 2G or 3G mobile connections for Internet access. An H.264-encoded 720p/30 fps stream at 3.5 Mbps would be reduced to 2.1 Mbps using H.264 with LCEVC, while, at 480p, a good quality experience could be delivered to cell phones at 1.2 Mbps.

One testing source confirming that MPEG’s criteria have been satisfied by the draft LCEVC specifications is the Streaming Learning Center operated by Streaming Media Magazine contributing editor Jan Ozer. Summarizing what he describes as “impressive” results, Ozer reports, “Overall, the analysis shows that if you substitute LCEVC with a base layer of x264 [the open-source software used for encoding H.264] for x.264 at 1080p resolution in a live use case, you should achieve about 40% bandwidth savings without noticeable quality loss, and you won’t need dedicated decoding hardware on the playback side.”

Ozer’s reference to a “base layer of x264” has to do with the fact that the basic AVC encode is set for a resolution, in this case 960×540, that is well below the resolution to be achieved with the LCEVC enhancement. The LCEVC processing closes the gap between whatever resolution is chosen for the base layer and the targeted resolution.

Rather than taking the MPEG linear macro-block approach to segmenting and encoding each frame in a video sequence, LCEVC supports a hierarchical technique using the parallel processing and scaling capabilities of today’s microprocessors to apply compression frame by frame. This is done in a way that’s analogous to how the human visual system brings a scene into focus as attention shifts or the scene itself changes within the broader field of vision.

Operating in real time, the processing begins by specifying the gist of the information associated with any pictorial element such as a face and then adds additional information to reach the required quality level for that element relative to quality reached in the uncompressed frame. The process determines how elements in the frame need to be reconstructed in strict adherence to variations across points of greater or lesser focus as captured in the original signal.

The elimination of inter-block prediction allows parallel processing of small 2×2 or 4×4 independent transform kernels, which is key to the dramatic improvement in encoding time as well as a reduction in use of processing power. As noted by David Ronca, director of video encoding at Facebook, eliminating barriers that have made HEVC, AV1 and other advanced codecs ill-suited for live streaming is vital to the future evolution of video services.

“The longer-term answer to video encoding cannot be to simply add more CPU capacity,” Ronca says in a recent Broadcast Bridge article about LCEVC. “This is an unsustainable model, both financially and environmentally. Codec research must emphasize both compression efficiency and computational efficiency.”

In the case of LCEVC used with HEVC, tests have shown that LCEVC, by minimizing processing power devoted to the core codec and shifting the rest of the processing to the enhancement layer, produces a net reduction of about 64% in total processing time compared to the time consumed by HEVC alone. With H.264, the encoding time, which in native mode is considerably less than that of HEVC,  is still reduced by about 58%. Presumably, though there’s been no test reports on this point, similar or even greater gains would be achieved with more processing intensive codecs like AV1 and VVC.

While it remains to be seen how encoding vendors will react to the availability of LCEVC, prospects for adoption in hardware-based products have been bolstered by commitments from two chipmakers. Xilinx, U.S.-based inventor of the FPGA (field programmable gate array) has already incorporated V-Nova’s technology into its chipsets, and Canadian SoC (system-on-chip) supplier NETINT Technologies, plans to introduce support for LCEVC in 2021.

Summarizing all these points, it’s clear there are at least nine reasons LCEVC provides content providers an opportunity to circumvent the codec wars.

LCEVC:

1. Increases compression ratios in the 40% range with any codec used in video streaming.
2. Maintains backward compatibility with whatever codec it is used with.
3. Can be implemented on devices via thin-client software or through browsers via HTML5.
4. Reduces encoding time that would normally be required using a given codec to achieve a targeted compression ratio by about 60%.
5. Significantly reduces use of processing power.
6. Will be supported in hardware by at least two chip suppliers.
7. Can be used with encoding for streaming on the Red5 Pro platform to support real-time transmission of advanced video formats.
8. Can make live streamed video available to devices on bandwidth-restricted 2G and 3G mobile networks.
9. Generates cost savings with reduced bandwidth consumption and the avoidance of new encoder costs.

Red5 Pro can be used with any hardware or software encoding platform. For distributors who rely on Red5 Pro to achieve industry-leading real-time streaming performance, there’s much to gain by requesting support for LCEVC from their encoding suppliers.

To share exploration of the possibilities, contact info@red5.net or schedule a call.