Big Gains in Efficiency, UX & Collaborative Options Impact Everything from Sports & Esports to Motion Pictures & Episodic Programming
Reliance on remote collaboration in high-value video production is proving more valuable than ever for producers who can source real-time contributions to their projects from any Internet-connected location.
The shift to location independence in production and postproduction isn’t just about reconfiguring old ways of doing things, which has accelerated during the coronavirus pandemic. Equally important, remote video production cloud collaboration is enabling development of new types of content, ancillary features and interactive applications that are transforming user experience.
Remote production strategies have been gaining traction for some time based on the strength of cloud-based workflow platforms enabling per-project groupings of widely dispersed contributors that would otherwise be impossible in a cost-sensitive, travel-averse marketplace. Until now, network-imposed lag times in video and other resource sharing have restricted what can be accomplished, especially with collaborative participation in sports, news and other live event productions.
Enabling real-time input of A/V and graphics content from any location for synchronized treatment by directors, engineers, editors and colorists in other locations is essential to achieve the cost efficiencies and other benefits remote collaboration brings to producing traditional content. Ultra-low latency is also fundamental to collaborations that go into creating new types of services and content enhancements.
Such possibilities require implementation of a secure streaming technology that can operate over the public Internet to transmit video at any resolution from any place to any number of people anywhere in the world within a few hundred milliseconds. That kind of performance is beyond the reach of most, but not all, streaming platforms.
As growing numbers of producers and vendors engaged in pushing the collaborative envelope are discovering, the requisite latency and other performance parameters are achievable with deployment of Red5 Pro’s cross-cloud streaming platform. In myriad ways they are demonstrating what can be done when remote production workflows are executed instantly and synchronously across all locations.
“The best cloud provider to solve latency is Red5 Pro, full stop.”
Todd Erdley, founder & CEO, Videon
With support for transmission speeds at well under half a second over any distance at any scale up to millions of simultaneous users, Red5 Pro enables remotely dispersed collaborators to share videos, graphics and communications with real-time efficiencies that once were only attainable inside a single location. “The best cloud provider to solve latency is Red5 Pro, full stop,” says Todd Erdley, CEO and founder of Videon, a leading supplier of compact encoders and streaming solutions used in production, postproduction and producer playouts to distribution affiliates.
By integrating Red5 Pro with its VersaStreamer encoders, Videon has provided its customers the option to instantiate ultra-low latency collaboration whenever needs arise, Erdley explains. “The days of bringing people into a studio to use fixed racks of equipment are gone,” he says. “Producers are moving micro elements of production to the field, which is where Red5 Pro with Videon really shines.”
At the same time, other entities are taking advantage of ultra-low latency to move the needle on content enhancements in ways that were never achievable with past attempts at interactive TV. OTT iTV applications targeted to generations of interactive-acclimated users are more personalized; the range of communications from text to video is broader, and feature input occurs in real time, creating new possibilities for sports, esports and other live events.
Some producers are taking advantage of extraordinarily low latency to incorporate betting and polling applications into live sports and other programming. Their viewers all see the same content and prompts for action simultaneously with no perceptible difference between when the action is viewed in person and when it’s seen online.
Other innovators are giving users direct control over some of the things that have typically been reserved exclusively for postproduction. For example, they’re taking advantage of stream synchronicity and bandwidth-efficient modes of distribution to give users the opportunity to curate their own viewing experiences by switching between streams from multiple camera angles.
Another benefit to user experience, which leverages both collaboration and ultra-low latency, comes with the ability to deliver personalized interactive graphics enhancements to live and on-demand content in tandem with device playback. Such capabilities are a focus of endeavors overseen by Andrew Heimbold, who serves as president of integrator Reality Check Solutions and helps lead the affiliated graphics tech platform supplier Singular.Live. Both companies are in the vanguard of feature-enhanced content development.
By delivering personalized graphics as overlays to the individual unicast video streams, producers can create a much richer viewing experience without having to accommodate network-based injection through just-in-time packaging on each unicast stream, Heimbold says. Instead, the overlays can be streamed separately for rendering with core content on user devices.
This requires a streaming platform that can be counted on to deliver the core video and overlay streams in precise temporal alignment. Achieving such capabilities at the ultra-low latencies required for sports viewing can be done with Red5 Pro, Heimbold says.
“From what we’ve seen, Red5 Pro is the most robust, scalable enterprise.”
Andrew Heimbold, president, Reality Check Solutions
“If you want a low latency you need a real time low-latency protocol,” he adds. “From what we’ve seen, Red5 Pro is the most robust, scalable enterprise, and they’ve been great to work with.”
As producers gain more insight into best practices,there are many aspects to orchestrating cloud-based collaboration in production and postproduction that remain to be normalized. However, many entities pursuing this paradigm shift have already discovered that the means of sharing multimedia input at hundreds of millisecond speeds from workstations hundreds and even thousands of miles apart is already in place.
The two-fold purpose of this document is (1) to provide an overview of market and technical developments that are unleashing collaborative and revenue-driving opportunities in production and postproduction and (2) to explore the range of possibilities in collaborative production that have been opened through implementation of ultra-low latency streaming on the Red5 Pro platform.
FORCES DRIVING THE MIGRATION TO REMOTE PRODUCTION
All aspects to producing network-delivered video programming are currently undergoing some form of transformation. Amid a surge in Internet streaming outlets seeking traction with an ever-more fragmented, and restless population of consumers, the need to produce increasing volumes of content in tight budget circumstances with enough bells and whistles to stand out from the crowd mandates a break with old ways of doing things.
The emerging production/post-production environment, facilitated by advances in production-related cloud solutions and other technologies, is one where speed of work order completion with the ability to collaborate in real time across dispersed locations is fundamental to success. Adding to the challenges, mass market embrace of OTT services has created opportunities to leverage IP technology to create much richer, more personalized enhancements to content that frequently must be executed in real time.
Surging Volumes of Original Content
Understanding the challenges to be met in shaping production strategies starts with an assessment of the impact consumers are having on the movement of content to the Internet. Viewers today are likely to be subscribing to at least one OTT service, whether or not they’re subscribers to traditional pay TV.
Worldwide, the global OTT subscription count, at 613 million as of YE 2018, was up by 100 million from the previous year and topped the cable subscription total by over 50 million, according to the Motion Picture Association of America. Frequently consumers opt for multiple OTT services with or without a traditional pay TV subscription.
In the U.S., where, as of 2019, close to 75% of broadband households were using at least one OTT service, more than half of those homes were signed up for two or more OTT services, according to Parks Associates. Globally, the average number of OTT subscriptions per viewer stands at 1.2 with OTT viewing hours averaging 6.8 hours per week, according to a report from CDN operator and Red5 Pro customer Limelight Networks. (See below)
Average Weekly Time Spent Viewing OTT Video
U.S. India Singapore Italy U.K. S. Korea France Germany Japan Global
8.55 8.43 7.62 7.35 6.49 6.38 6.05 5.52 4.80 6.80
Average Per-Household Subscription Counts
1.7 1.6 0.9 1.3 1.3 1.1 1.1 1.1 0.9 1.2
Source: Limelight Networks
A Global Phenomenon
These trends have had an immense impact on production workloads, starting with the fact that competition for viewers has produced an unprecedented explosion in original program development, including episodic programming patterned on the early successes of productions from Netflix and Amazon. Along with spawning ever more commitments to such content from OTT competitors, consumer responses to these efforts have triggered a surge in commitments to new programming from traditional TV producers for both their legacy TV outlets and the direct-to-consumer (DTC) services they’ve launched in increasing numbers over the past two years.
The original production trend is touching every part of the globe. In the U.S., taking movies as well as TV and streamed video content into account, Variety reported the total spent on original productions by U.S. firms in 2019 came to $121 billion, led by Disney’s $27.8-billion investment, a significant share of which went to product for its recently launched Disney+ online service. Both HBO and Netflix expanded the volume of original content produced in 2019 by 50% over 2018, Variety said.
Pandemic-imposed obstructions slowed the pace to some extent, but accelerated adoptions of remote production strategies seemed likely to mitigate the impact. In any event, sooner or later the aggregate original production investment was sure to keep climbing with NBCU’s Peacock, AT&T’s HBO Max and other new OTT services coming online in 2020. Apple alone said it had committed to investing $6 billion in original programming for the recently launched Apple TV Plus.
The U.S.-based OTT giants are not alone. In France, for example, the top five broadcasters, after raising spending on original content to over 40% of their combined €5.4 billion programming budgets, had 106 new local shows in the pipeline as of mid-2019.
In the U.K., Sky has committed to a doubling of its original content investment with the launch of a new production facility to serve all its European outlets. Another especially noteworthy case, Eastern Europe’s OTT market leader MEGOGO has gone so far as to launch its own production studio to generate 300 hours of programming in its first year of operations with ambitions to ramp to 1,000 hours by 2022.
The country with the second largest amount of money being spent on original programming is China. As of 2017, $4.5 billion had been earmarked for OTT outlets and $6.4 billion for broadcast and pay TV distribution, according to IHS Markit. In Southeast Asia, regional SVOD provider HOOQ committed to having 100 original productions in its lineup by Q2 2020, including 12 new TV series and seven films.
Overcoming the Cost, Talent and Time Squeezes
Against this backdrop, low profit margins stemming from the intensity of competition are forcing producers to find ways to spend less on their projects even as the competition forces them to produce more content. Remote collaboration on production and postproduction projects offers a way out of this bind.
The ability to include remotely positioned contributors in production and post production workflows frees producers to hire people as needed on a per-project basis without incurring large travel expenses to bring them into central production facilities. Along with lowering permanent staffing costs, this makes it easier to find enough qualified contractors to handle the required workload as competition for talent shrinks the work pools in major production centers.
Another major aspect of the new original production reality is the reduction in time allotted to completing projects. In the past when production schedules were less packed and ran on seasonal cycles, production on a TV series would start ahead of set launch dates, leaving plenty of time for post production on pilots, assessment of reactions from user groups and fixing of production and postproduction schedules for each episode.
Today, there is a persistent, year-round need to feed the OTT market with no predictability as to when the next uptick in workloads will occur. Additionally, while set theatrical release dates are still a major aspect of filmmaking, scheduling must be flexible enough to accommodate all the films entering the pipeline for OTT distribution that may or may not be tied to theatrical release dates.
The immediacy underlying the need for new content as providers battle for competitive advantage has significantly reduced the time available to get the job done. Everything must be accomplished faster than ever before. Consequently, even as the cost squeeze and the need to find enough people to handle all the workloads accelerates the transition to remote collaboration, work now done across widely dispersed locations must be completed faster than ever.
Live Streaming Productions
Compounding the impact of original programming production workloads on the transformation in production, the surge in OTT streaming of live sports, esports and other linear content, including events with niche followings large enough to merit attention, mandates a much lower-cost approach to live production. Here, again, remote production strategies that can eliminate major costs with no loss in production speed and quality are key.
Sports, which shut down with the advent of the coronavirus pandemic, by all accounts have been the biggest force behind live-stream viewing. They are sure to resume that role as live video streaming traffic continues on course to exceed on-demand video traffic by 2023, as predicted by Rethink Research.
Sports channels are now a routine component of virtual MVPD services like Fubo TV, YouTube Live TV, Hulu Live TV and others, while ever more special sports league packages have become mainstays of online viewing across the globe. At the same time, demand for online access to live sports has brought other types of Internet firms into competition for live streaming rights, including Amazon with its multi-year Thursday Night NFL and English Premiere League (EPL) deals; Facebook with rights to Major League Baseball, Major League Soccer and a variety of college and international sports events, and Verizon with an NFL deal that has made games available on its AOL and Yahoo sites.
Unprecedented consumer enthusiasm for online access to sports events was evident everywhere as users continued to break streaming records prior to onset of the pandemic. For example, Fox Sports, producer of the 2020 Super Bowl, said its average per-minute online viewer count at 3.4 million far outstripped previous Super Bowl averages. The previous fall, NFL games streamed by Fox caused spikes in global video traffic reaching as high as 29% and averaging 4%, according to advertising metrics supplier Conviva.
Other events breaking their streaming records in 2019 included the Rugby and Cricket World Cups, basketball’s March Madness, the U.S. college football championship, U.K. and other European soccer league finals and many more. According to online video publisher Ooyala, more than 45% of sports viewing in the U.K. and Ireland occurs on smartphones and tablets.
Meanwhile, the opportunity to reach niche audiences online has opened the floodgates to streaming of dozens of minor sports and other types of competition that wouldn’t merit allocation of channel space on legacy pay TV networks. Low-cost, professional-caliber remote production capabilities have made the likes of Segway polo (played on Segway personal transporters), drone racing, curling, surfing and international dodgeball championships available to viewers worldwide.
Adding to the acceleration in live productions, esports events have reached digital audience levels on par with traditional sports. In 2019, per-event viewership averaged 480 million, representing an 18% increase over the previous year, according to Futuresource Consulting. Total time spent by all viewers streaming esports worldwide came to 5 billion hours, which led Netflix to identify esports as a bigger source of competition for viewership than any of its SVOD competitors.
Service Differentiation Through Dynamic Feature Enhancements
Demand for personalized interactive feature enhancements to sports and other linear programming as well as content accessed on demand represents still another force behind new approaches to production. Where multichannel operators once controlled user experience (UX) on their own terms, a combination of market forces unleashed by OTT entrants riding advances in broadband, device and cloud technology has pushed consumers to expect more choice with much greater flexibility in where and how they consume video and related services.
This sea change was documented by a survey of 3,100 U.S. consumers conducted by TiVo’s Digitalsmiths unit, which found that the personalization of UX was nearly as important as price in determining people’s choice of video services. When asked to name features they found appealing, 59% of the respondents chose price, 55% chose personalization of their UIs and 42% chose recommendations.
This is a worldwide phenomenon, as revealed in a global survey of 2,000 consumers conducted by 451 Research Group, which reported 55% of respondents said they value the ability to control and personalize their own content and viewing experiences. Notably, 49% across all age brackets and 54% in the 18-34 millennial category indicated they expect their video service providers to use the data they collect to provide custom-tailored service offerings and customer support.
Consequently, what had mostly been a shared, passive household approach to video consumption is becoming a highly individualized experience where every user has an opportunity to shape and socialize what they consume on their personal devices. With workflows running on cloud infrastructure, video service operators of all stripes, including content providers pursuing DTC offerings, can extend personalization beyond the usual recommendations to include social connectivity, app-specific messaging, ancillary video clips, choices of camera angles, betting and polling options, purchasing tied to products appearing in the story or game flow and much else.
Some of these features are intrinsic to the service apps, but others are based on real-time insertions of text, graphics and images with the packaging process in unicast streams. The latter represent a vast area of largely untapped opportunity for distributors to differentiate and enhance the appeal of their services.
For example, the opportunity for viewers to choose among multiple camera angles as they’re watching live sports, concerts and other events, which has been discussed since the early pay TV years, is now doable at mass scale with OTT services. Other approaches to personalization can deliver content specific to the user’s interests within the primary video stream.
A sports network can give fans the option to open a window in the display that streams clips of their favorite players’ latest exploits or conveys how their fantasy sports teams are doing in real time on game day. Business networks can spin up personalized streams of news and stock performance related to viewers’ investment interests. Indeed, personalized “channels” delivering clips relevant to viewers’ entertainment, news and other interests could eventually become a common feature of OTT services.
Amid the surging popularity of online betting, sports leagues of every description have joined traditional betting venues like horse racing in the search for compelling applications that heighten audience engagement. Online betting abetted by real-time streaming offers fans an experience much more akin to betting on premise than online betting apps that rely on traditional, higher-latency streaming modes.
These are early days, but those who have such capabilities will certainly find ever more ways to use them to their advantage. The momentum behind this new realm of personalized user experiences will inevitably lead to a next-generation OTT service infrastructure that frees providers to pursue meaningful new opportunities wherever they find them.
Technology Support for Remote Production Workflows
Industry supplies have responded to these developments with an outpouring of solutions supporting the remote video production cloud collaboration in production and postproduction that’s essential to moving the market forward. As a result, producers can now engage in-house personnel and specialists operating as independent contractors whether they’re across town from each other or a continent apart with confidence their collective output will be on par with traditional single-location operations.
Aggregation of Cloud-Based Solutions
Cloud technology is the pervasive lingua franca in this transformation. Cloud-based platforms covering every aspect of production and postproduction have been architected to exploit the most efficient virtualization frameworks, making it possible to seamlessly leverage private and public cloud facilities in whatever configurations work best from one moment to the next.
Partnerships, sometimes involving dozens of vendors, have created the integrated cloud frameworks essential to enabling remote collaboration on workflows encompassing all aspects of the production and postproduction processes. These systems are tasked with making sure content is where it needs to be in the format it needs to be in for playout to every type of affiliate.
- Integrations with storage platforms allow participants in the production workflows to extract what they need for archives.
- Integrations with content and media asset management systems allow technicians to keep track of all their A/V assets, what formats they are in and all the metadata associated with those assets.
- Protection mechanisms, including digital rights management (DRM), watermarking and other processes, can be assigned to each piece of content on the fly.
- Advertising assets can be pulled in through interaction with campaign management platforms and ad servers.
Workflow automation that streamlines access to all these functionalities from all participants’ workstations is another fundamental component in the transformation to remote collaboration. Automation is essential to avoiding a wide range of issues: inconsistencies in file naming, packaging and encoding, inefficient use of storage with duplicated and over-sized files and the burdening of specialists with counter-productive time constraints as they wait for workstations to perform transcoding, framerate conversions, captioning and other processes.
Such issues are mitigated through reliance on common trigger interfaces utilizing API integration, XML support, watch folders and other measures comprising today’s best practices. Two of the most important standards contributing to production and postproduction automation are the Material eXchange Format (MFX) and the Interoperable Master Format (IMF).
MFX serves as a container for different streams of coded essences in non-linear exchanges of digital files over IP links for use in production and postproduction processes. When MXF files are used to wrap media content with useful metadata at the beginning of the production process, that metadata can be accumulated, processed and expanded to include information about asset management, digital rights management and media archive systems all the way through to distribution.
IMF enables creation and management of different versions of the same video essence in postproduction by repurposing MFX and adding ports for integrating track files into a single flow. The protocol supports point-and-click creation of Composition Play Lists (CPLs) to describe the components in terms of video format, audio version, language, captioning, etc. that comprise each version of the program. These are packaged with Output Profile Lists (OPLs) to describe how different output versions are to be transcoded from the master essence, including instructions for codec formats, picture resolution, HDR modes, frame rates and audio versions.
MOVING TO THE NEXT LEVEL IN VIDEO PRODUCTION
Developments as described in Part 1 have brought the premium video marketplace to a new plateau where the key to exploiting new opportunities lies with eliminating latency as an issue in production-related operations. Cost-saving gains in production efficiency, new features and new types of services are all part of this new prospectus.
Of course, given the laws of physics, eliminating latency as an issue doesn’t necessarily mean eliminating latency altogether. Once latency becomes imperceptible in production collaborations over great distances, producers have what amounts to a real-time operating environment to work in to accomplish things that were either not doable before or were done with less than stellar results.
For example, in terms of executing basic steps in production and postproduction with a dispersed team of collaborators, the ability to transfer video-rich files between workstations, retrieve them from storage for inclusion in a production or upload them from remote locations to the cloud over the Internet at speeds commensurate with location-based transfers poses a significant challenge. Bi-directional transmissions of video, audio and metadata files at ultra-low latency neutralize distance barriers to these routine aspects of traditional operations in production and postproduction.
Fulfilling the REMI Promise
There’s also a much sought-after non-traditional aspect to production, commonly referred to as REMI or Remote Integration Model, where the ability to transfer video at ultra-low latency has revolutionary implications. This is especially true for costly sports and other live productions but also applies in any situation where producers can benefit from reductions in equipment and crew at remote filming locations.
Using home-base production facilities to process video flows from the field without having to send big production vans or pay for costly, latency-burdened satellite uplinks introduces new levels of versatility with faster turnaround as well as significant cost savings. “IP-based OB (outside broadcasting) trucks can arrive on location to cover even a large sporting event with a minimal team,” notes colorist and production workflow specialist Richard Lackey, in a post hosted by Cinema5D, a publication devoted to cinematography. “While the truck is plugged into the network, provided with dedicated high bandwidth connectivity, an entire event can be produced and operated remotely from a location across the country, or even on a different continent.”
The ability to work in real time is essential. “When you can get video from an event to a production center in less than a second, there’s a lot you can do,” says Todd Erdley, who, as mentioned in the Introduction, leads Videon, a leading supplier of encoders and other systems used in production and postproduction.
“We’re seeing [REMI] projects centered on all the major sports activities as well as at other levels of interactive sports,” he adds. “We’re seeing projects associated with fitness, trade shows, you name it.”
Latency-free REMI can include adding feeds into final production from other locations apart from where the event takes place. In the case of sports or esports, remote commentators, encapsulated in picture-in-picture videos can provide analysis in different languages or commentary with the ability to mix in localized graphics, social interactions and other material.
Integrating sports commentary from remote locations is something NBC Sports has begun now that there’s no latency barrier, reports Chris Connolly, senior director of transmission engineering of NBC Sports, in a video recorded during the NAB’s fall 2018 show in New York. “Commentators can see each other and play off each other and their reactions without just hearing it in their ears,” Connolly says.
The New UX Paradigm
Another new development taking shape with remote collaboration in production workflows entails the utilization of new graphics platforms to expand creative participation, range of enriching features and degrees of interactivity and personalization with all types of video content, including sports. The most compelling instances occur when personalized features curated independently in the cloud can be delivered as overlays in sync with the primary video streams at ultra-low latency for combined rendering on devices.
One dramatic example of what can be accomplished in collaborative sports productions occurred with the OTT live streamed version of Fox Sports’ 2018 U.S. Open Golf Championship coverage, which operated independently from the linear TV stream. Through a collaboration involving cloud-based video platform Grabyo and live graphics platform Singular.Live, Fox gave viewers on FOX Sports GO and the FOX Sports App complete control over which hole and players they could view from live feeds covering action at five holes throughout the tournament. Remotely located production personnel, aided by automated processes, fed graphics, stats and other data tuned to play in real time with each feed.
More recently, in May 2020 with the first instance of a return to top-tier European football league play, Sky Deutschland complemented its broadcast of Germany’s Bundesliga competition with a live-streamed “Fan Engagement” feed featuring graphics enhancements curated by graphics operators working from remote locations. Along with contributing to the safety of personnel amid the ongoing pandemic, the strategy cut the costs of bringing the graphics operators and all their gear to the game sites or Sky’s studios.
This was another case of remote graphics production involving use of the Singular.Live platform, which, by virtue of enabling browser-based control over creation of rich graphics overlays in real time, has been a leading force in leveraging remote production to bring a new level of enhancement to UX. When Singular.Live is integrated with a live video production suite like Grabyo’s, “you can trigger input of graphics from anywhere in the world,” says Andrew Heimbold, who, as noted in the Introduction, serves as president of Reality Check Systems (RCS) and helps lead Singular, in which RCS is a major investor.
As described by Heimbold, RCS is working with leading sports leagues, networks, federations and social media outlets worldwide to drive audience engagement through customized fusions of dynamic graphics, real-time data and social media. Many of these applications take advantage of the fact that, as an HTML5 platform, Singular.Live can render graphics, video clips, text and other elements delivered as an overlay stream through any HTML5-enabled browser, Heimbold notes.
“The benefit of working in HTML5 is it works on any device that can render a web page,” he says. “If the device supports rendering from HTML5, it supports Singular natively.”
The most common approach to delivering the graphics, typically displayed in the lower third of the display, is as a separate stream for rendering with the video on the device. As a result, the aggregation of features from the remote production sources can be personalized for each viewer.
“You can generate graphics in real time and distribute them anywhere,” Heimbold explains. “You can render them in a broadcast feed or on a million devices. In some cases, everyone might have the same experience, but the language used with the graphics might vary or you might want to deliver a personally targeted ad with the graphics overlay.”
Invigorating Interactivity through Real-Time Viewing
Interactivity tied to real-time viewing of events as they transpire on the field greatly enhances the appeal of polling and betting applications that are aimed at boosting consumer engagement in services. In the case of polling, audience responses to questions asking viewers to gauge performances of athletes can be posted in tandem with the rated action, as illustrated with streamed coverage of the annual Red Bull Rampage mountain bike competition near Zion National Park in 2018. The digital overlay enhancement enabled by Singular.Live offered viewers opportunities to pick their favorite rider, event and trick with continuous updating of polling results throughout the 4 ½ hour competition.
With the surge in online betting, there’s much at stake when it comes to delivering the superior UX enabled by real-time streaming. In one case in point, one Red5 Pro customer and major North American producer of horse races is enabling the real-time viewing of action at the track and up-to-the-second tabulations of bets and odds that are vital to eliminating the disparities between onsite and online betting experiences.
The online betting service allows bettors to monitor last-second news pertinent to betting choices and to see what’s happening in the paddock and as horses approach the starting gate. The company has built a cloud-based workflow that seamlessly unifies its back-office operations with onsite capture, encoding and streaming out to anyone who signs up for the service. Because everyone is seeing everything at the same instant, bettors can engage with each other on social media before, during and after races. More importantly, bettors can know the results of the races and their placed bets as soon as they happen live, rather than viewing the race many seconds afterwards.
New Horizons in Esports Production
Such capabilities are gaining traction along with unique applications of REMI in esports, where the costs and complexities of live multiplayer competitions along with demand for better user experiences are driving production initiatives. REMI backed by ultra-low latency can have a big impact, even though there’s no avoiding the need for large crews and the equipment they use at gaming venues to deliver the action to big screens watched by thousands of fans onsite as well as to vast audiences of Internet-connected and, sometimes, broadcast TV-connected viewers.
For example, Blizzard Entertainment, which produces multiplayer events tied to its Call of Duty League and Overwatch League titles in many parts of the world throughout the year, has created a production workflow that pulls multiple live feeds from onsite cameras for post production by editors at its Irvine, CA facilities. Complicating matters, Blizzard employs several rather than just one onsite “observer” or camera operator to capture the field of game play in tandem with video captured by cameras focused on the gamers, creating a complex mix of videos that have to be composited into the live version seen by onsite and remote viewers.
The post process also involves pulling graphics, hype reels, sponsor messages and other content from various sources for compilation into the graphics stream that runs with the main video.
The primary video and graphics outputs from Irvine are transmitted back to the live venue for onsite viewing and out over the Internet to end users.
All of this has to happen fast enough to ensure that what onsite viewers see in the composite stream is as much in sync with what they’re seeing and hearing off screen at the venue as would be the case if the post processes were done onsite. Another set of editors in Irvine delivers versions to archives for VOD access and creates highlight clips for various social media and other digital platforms.
Similarly, Riot Games, producers of League of Legends esports competitions that reach tens of millions of viewers worldwide, has steadily streamlined its operations to enable substantial portions of the production workflow to be handled at its studio in Los Angeles or regional studios in other countries, even though the venue might be thousands of miles away. As explained by Riot’s Esports Technology Group in a late 2019 blog, multiple feeds from cameras and crowd microphones are transmitted to the production center where everything is decoded, transported to relevant control rooms and processed for various streams going out around the world.
A core world feed comprised of shots and instant replay is created without audio, which is added in multiple languages before transmission. The company has served as many as 99.6 million concurrent viewers with versions in 19 different languages.
“The footprint has decreased from having several production trucks onsite with all the operators and creative personnel, alongside several satellite trucks, to today’s iteration consisting of about 10 high density video, audio, encoding, and network racks in one onsite engineering room,” the group says. “This is all coordinated via our real-time, global communications infrastructure, spanning several continents and time zones, including the flagship studio in Los Angeles and partner facilities in places like Brazil, China, Korea, and Germany.”
The Ultra-Low Latency Solution for a New Era in Remote Production
Such are the capabilities that can be attained when remote collaboration on video production is supported by virtually latency-free transfers of video, graphics and other assets. The cross-cloud live streaming platform provided by Red5 Pro makes this possible.
It doesn’t matter how far apart participants in the production and postproduction workflows might be or how many might be involved. When Red5 Pro is in play, everyone will be working in the same real-time temporal space where shared experiences can be synchronized with accessibility to transferred assets within a fraction of a half second of transmission from any source.
Red5 Pro is designed to support video streaming over the Internet in any situation where the elimination of perceptible lag time is essential to successful outcomes, regardless of whether just one or millions of recipients are involved. When employed as an alternative to traditional CDNs in streaming live content to consumers, Red5 Pro software is instantiated in multiple private and public cloud facilities to deliver video through a hierarchy of origin, relay and edge nodes utilizing whatever combination of major transmission modes serves a distributor’s needs.
Supported protocols include WebRTC (Real-Time Communications), RTSP (Real-Time Streaming Protocol), RTMP (Real-Time Messaging Protocol), MPEG-TS and SRT (Secure Reliable Transport). WebRTC is the most commonly used protocol in Red5 Pro implementations and is best suited for meeting the ultra-low latency requirements in the types of real-time production and postproduction scenarios described above.
Of course, there are many situations, typically when production of sports and other live content isn’t involved, where other modes of file transfer with higher latency profiles are appropriate for use in collaborative projects. This is a point Videon has taken into account with its support for three streaming options that have been integrated with its VersaStreamer encoders.
As noted in the Introduction, one of those options is Red5 Pro. The others are SRT and conventional HTTP-based ABR streaming, which comes into play when the VersaStreamer encoder is used with a Videon CDN or public cloud partner.
“There’s a range of different needs for low latency, including some that need the real-time connectivity enabled by Red5 Pro,” Todd Erdley says. “You can achieve 150-300 ms. latency with Red5 Pro. It can be a second with SRT and 2-3 seconds with companies like AWS and Akamai.”
The versatility of the Videon encoders and their significant contribution to lowering latency in the encoding process rests in large part on the fact that the compact purpose-built units run on Qualcomm’s high-performance Snapdragon processors. “This is the only encoder in our field that uses Qualcomm hardware,” Erdley notes. “The key thing is the encoder can be used in all three types of workflows involving these different levels of latency simultaneously.”
The release of SRT as an open-source platform in 2017 after its development as a proprietary protocol by Haivision is widely credited as a major factor in the industry’s shift to collaborative, location-independent production and postproduction. But latency parameters that worked in that timeframe don’t always work for today’s requirements, which is why Videon took the trouble to add Red5 Pro-based streaming as an option with its encoders. The ever-more restrictive latency requirements taking shape with the surge in live-streamed sports and esports has created an environment where the difference between 1-second and 150 ms. latency really matters.
Along with much lower latency Red5 Pro introduces another benefit that’s important to the dynamism in remote production. Because WebRTC, unlike SRT, can be played from any of the major browsers, including Chrome, Edge, Firefox, Safari and Opera, without the aid of plug-ins, Red5 Pro can be instantly implemented wherever the need arises for input from a new location.
Moreover, video packaged for streaming on Red5 Pro among dispersed locations in the production process can be directly output for streaming to millions of end users in instances where the platform is set up for that purpose, eliminating any repackaging delays or the need for new device player software.
Pushing the Envelope in a Space Age Use Case
The search for a streaming platform that can accomplish these things within a closed network connecting thousands of employees in multiple locations has been a priority for a leading commercial operator of space missions for governments and industry. Two basic requirements are motivating the search, says an executive who, for security purposes, requests anonymity for herself and the company.
“More than anything, we need a stable platform,” the executive says. “With thousands of people watching hundreds of feeds, you can end up very quickly with ten, twenty thousand concurrent views. I need a platform that can scale and grow with that, that won’t topple over as you expand the network and increase the load on it.”
“When that clock hits zero, you want everyone to see the rocket lift off at the same moment.”
Commercial Space Transport Operator
The second most important thing is latency. “You no longer feel like you’re part of a shared experience when you’re watching something out of sync with the person sitting right next to you,” she says. “We have a general temporal anchor in the form of that countdown clock. When that clock hits zero you want everyone to see the rocket lift off at that moment.”
And with all eyes remaining on the flight after liftoff, it’s imperative that everyone can see and be able to communicate about what’s happening in as close to real time as you can get with a camera feed that is ascending into near space. When the signal reaches ground it must be terrestrially streamed for simultaneous reception within 200 ms. or less at observation posts across the U.S.
The company hasn’t yet replaced its existing streaming infrastructure with Red5 Pro. “But we’re taking a very serious look at it,” the executive says.
The same can be said for the commercial video industry as a whole. As spelled out in the preceding pages, the next steps in the evolution of remote collaboration on production and postproduction of high-value content depend on the capabilities uniquely supported by Red5 Pro.
Remote collaboration is fast becoming the new norm in high-value video production as producers discover it’s now possible to initiate real-time participation in their workflows from any point of Internet connectivity.
Many market trends have converged to drive industry-wide adoption of remote production strategies. As consumer demand for Internet access to premium content draws ever more OTT providers into the crowded field, the volume of original productions surges, creating cost, time and staffing pressures that can only be met through greater reliance on contributions to production and post production workflows from dispersed locations.
Similarly, demand for live-streamed sports and esports has triggered a surge in live online productions that requires use of remote production technologies to eliminate excessive costs. And the opportunity to differentiate all types of services through personalized feature enhancements has drawn producers to use of remote production platforms dedicated to such applications.
These market dynamics call for the fullest possible exploitation of the possibilities in remote production, which until now has been limited by latencies incurred with sharing of videos and other large files across dispersed workstations and with the importation of content from remote locations. These restrictions no longer apply when video can be streamed with no discernable delays over the Internet from any point to another in production and post production workflows, no matter how far apart those points are.
This is why ever more producers are taking advantage of Red5 Pro’s cross-cloud streaming platform, which is designed to achieve the lowest possible latency whether it’s used in production workflows, contribution playouts or distribution to millions of viewers. With utilization of Red5 Pro:
- Producers are benefitting from major cost savings as they cut travel, equipment and staffing expenses by contracting with specialists to join their workflows from any location on an as-needed basis.
- Real-time connectivity of production facilities to remote points of video capture is lifting budget constraints on everything from major sports and esports productions to productions of less popular events that would otherwise be unavailable to passionate niche audiences.
- Producers are creating far more compelling user experiences ranging from dynamic graphics insertions to betting and viewer-curated UX.
In other words, by coordinating workloads in real time no matter where contributors are located, producers are opening what promises to be a new era in remote production.
To learn more about how Red5 Pro can facilitate latency-free applications of remote production, feel free to email firstname.lastname@example.org or schedule a call.