Smart TV App Development Enters Real-Time Streaming Domain

A sudden turn of events in smart TV development is freeing service providers to bring real-time video interactivity into play for breakthroughs in user experience on home entertainment’s new center stage.

Comcast kicked off the potentially disruptive trend in early October when it announced that Sky, its recently acquired UK-based subsidiary, was introducing a smart TV labeled Sky Glass with a feature-rich 4K camera developed by Microsoft. Sky Glass viewers will be able to stream pay TV services that once were only available via satellite while interacting with other viewers remotely via video communications.

As described by Sky officials, the Sky Glass camera is analogous to the Xbox Kinect, a device that was pulled from the market for a variety of reasons having to do with strategic missteps in the gaming market. The Sky Glass version is designed to seamlessly interact with features supported by the smart TV, including video chat, watch parties and an expanded portfolio of Sky-delivered single-player and social games that can be played with hand gestures tracked by the camera. The camera will also provide support for a fitness app by reacting to users’ execution of exercises as they’re guided through their routines.

These and other new service opportunities emerging with original-equipment-manufacturer (OEM) deals like Sky’s stand to make service providers stronger competitors in the OTT-dominated premium video marketplace, especially if they move away from sole reliance on one-way HTTP-based streaming technology. By exploiting the power of interactive real-time video streaming as supported by Red5 Pro’s Experience Delivery Network (XDN) platform they can deliver a wide range of attractive services and features that have never been available on the primary home viewing platform.

Comcast Ignites Global Trend

The Sky Glass partnership with an unnamed OEM is just one of many such arrangements that have suddenly gone public in the past few months. In all cases, the self-branded smart TV sets empower legacy pay TV providers to control content and applications in ways that haven’t been possible with TV sets that are under the OEMs’ direct control.

Subscribers who buy these sets can stream full line-ups of cable or satellite services along with special features and affiliated over-the-top (OTT) video services without connecting to traditional set-top boxes or internet media players. Service providers are incentivizing purchases with low pricing and interest-free monthly payments added to subscription billings.

Adding to the potential impact of the Sky Glass initiative, Comcast said that along with launching the package across Sky’s service areas in six other countries next year, it was making the platform available to other service providers as well. Australia’s Foxtel Group is the first announced affiliate with plans to begin delivering its market-leading pay TV service over Foxtel-branded smart TVs in 2022.

In addition, Comcast quickly followed the Sky Glass announcement with word that it is making its stateside Xfinity TV services available to anyone purchasing a new line of XClass smart TVs produced by Hisense. While Comcast didn’t mention whether the TV sets would be equipped with cameras, it broke new ground on another front by letting it be known that people who buy the displays will be able to watch its Xfinity TV services regardless of whether they reside in Comcast Xfinity service areas.

Also in October, Vestel, the Oslo-based smart TV manufacturer for JVC, Panasonic, Hitachi, Telefunken, Toshiba, and about 100 other brands, said it had teamed with OTT software provider Vewd to create an Operator TV platform. This allows service providers to forge smart TV deals with any Vestel OEM affiliate.

It’s also worth noting that in September Amazon began offering two lines of smart TVs under the Amazon Fire TV brand to all consumers, whether or not they’re subscribers to Amazon Prime. Their aim is to offer customers the unique benefits of an in-house approach to TV integration with Alexa in support of voice activated access to streaming apps which, up to now, required the use of an Amazon Fire TV Stick.

Real-Time Streaming Opens New Path to Developing Smart TV Apps

These agreements come as OTT providers’ increasing reliance on delivering compelling user experiences with live content has sparked a pervasive need for real-time connectivity that ensures all viewers see the same thing at once with latencies in the 200ms-400ms range or even lower. When it comes to reaching smart TV viewers, this is the case whether or not the set is equipped with a camera.

Real-time connectivity is also required in all instances where simultaneous reception of live video content across mass audiences is essential to interactive engagement, even if their input is text-based or delivered via video from cell phones or other connected devices. Whether people are responding to live-streamed shopping pitches or a disputed ruling in a football game, everybody needs to see the same thing at the same time.

That said, there’s an explosion of opportunity related to bringing video-based interactivity to the TV set. This is aided by the fact that many of these OEM partner initiatives are opening a user-friendly path to using webcams with smart TVs to support watch parties, video conferences, socialized game playing, and other interactive video experiences that have taken off across the internet.

Moreover, as described in this blog, normalizing interactive video in the viewing experience is likely to spark new types of traditional programming. Game shows, reality TV, late-night variety, shopping channels and other live TV programs can all benefit from remote audience participation via video.

New Thinking about Facilitating Use of Webcams with Smart TVs

Up to now, it’s been hard for content producers and service providers to foster development of applications requiring use of webcams owing to smart TV OEMs’ decisions several years ago to stop bundling webcams with their displays. This came in response to widespread concerns over users’ potential vulnerability to violations of their privacy. People with the right smart TV portals could plug in separately purchased webcams, but they were on their own when it came to setting them up for their individual needs.

Now service providers who may shy away from embedding webcams with every self-branded smart TV set can specify webcams that work with a variety of new use cases that can be easily accessed by anyone that wants them. For example, Comcast said the new Hisense XCast models for the U.S. market, now sold through Walmart, are built on the template used by Spy Glass with similar display, sound, and portal parameters. That means that even if the sets are sold without webcams, the platform is designed to enable integration of any cameras the company might eventually choose with activation of interactive video features like those enumerated by Sky.

Comcast and the other entities pursuing the smart TV strategy appear to be all in with the belief that a better smart TV viewing experience will add significant appeal to their service packages.

The voice-controlled Sky Glass platform has been integrated with several streaming services, including BBC iPlayer, Netflix, Disney+ and others.

With prices ranging from £649 to £1,049 for displays measuring from 43 to 65 inches, all Sky Glass TV models support the leading HDR modes, deliver Dolby Atmos surround sound through six speakers, and connect with other devices through one USB-C and three HDMI 2.1 ports. Officials said they will continually add functionalities along with new apps and features in future models that buyers of the first generation will be able to trade for at no additional cost.

As for the new Hisense XCast models for the U.S. market, providing a means by which people anywhere can access Xfinity TV gives the operator a far larger potential subscriber base. The displays, measuring 43 or 50 inches, are bargain priced at $298 and $378, respectively.

Like Sky Glass, the XClass app package will feature an eclectic grouping of streaming apps, including Comcast’s Xfinity and Peacock apps, Hulu+, YouTube TV and Sling TV. The streaming app from fellow U.S. cable operator Charter Communications will also be in the mix.

Vestel and Vewd offered fewer details about their plans in an announcement that came out ahead of any known deals between Vestel Smart TV affiliates and service providers. But officials stressed their goal is to support whatever functionalities and features a highly diverse range of pay TV partners might need, presumably including anything related to the use of webcams with their smart TVs.

In contrast, Amazon was specific about the video communications possibilities in its smart TV announcement, noting the Fire TV Omni and 4-Series models are designed to enable video calling with whatever brand of qualified webcam a customer chooses. The first step in this direction will be the launch of the Zoom video calling app on the Omni by year’s end.

The Amazon smart TV content portfolio mirrors the content available on Amazon Fire TV Sticks, which includes over 400 channels from OTT services such as YouTube TV, Sling TV, Tubi, Pluto TV, Philo, Prime Video Channels, Prime Video Live Events, and many more. Such a vast live channel lineup provides a great environment for bringing video communications into the viewing experience.

To what extent Amazon welcomes efforts by its affiliates to offer future “watch party” or other video communications apps that might be integrated with their programming remains to be seen. But Amazon’s as well as Comcast’s recognition that support for real-time interactive video applications is a vital component of the emerging TV viewing experience suggests it’s just a matter of time before there’s a smart TV user base to be targeted by innovative uses of interactive real-time video streaming infrastructure.

In fact, with a broad base of webcam-connected smart TV users at hand, any live OTT video service providers whose apps are featured with those displays would be in a position to implement these new audience engagement strategies in conjunction with streaming support from XDN infrastructure. Moreover, for players like Comcast who control the platforms underlying the new smart TVs, the opportunities extend beyond entertainment to making the TV a core component in other aspects of household life where real-time video interactivity can be a major contributor to user experience.

As Comcast CEO Brian Roberts said in a recent earnings call, the smart TV is a “platform for innovation on a go-forward basis for where we think television will evolve, whether it’s gaming, whether it’s fitness, healthcare or education. Having that be part of your relationship with our company is novel territory for us to do R&D off of.”

The Practical Approach to Activating Real-Time Streaming for Smart TV Apps

As described in greater detail in this white paper, everything essential to enabling multidirectional live streaming in real time from primary sources and any number of users is embodied in XDN architecture. The platform achieves the requisite scalability, quality, and other performance parameters with fail-safe redundancy through automated orchestration of hierarchies of Origin, Relay, and Edge Nodes in one or more cloud clusters.

XDN architecture makes use of the Real-Time Transport Protocol (RTP) as the foundation for streaming via WebRTC (Real-Time Communications) and Real-Time Streaming Protocol (RTSP). In most cases, WebRTC is the preferred option for streaming on the XDN platform owing to the fact that it is supported by all the major browsers, which eliminates the need for device plug-ins. This means any user accessing live streaming through Chrome, Safari, Firefox, Edge, or Opera can also participate in interactive video communications by sending their input over the XDN via WebRTC.

There are also other options for receiving and transmitting video in real time when devices are not utilizing any of these browsers. RTSP, often the preferred option when mobile devices are targeted, can be activated through Red5 Pro iOS and Android SDKs. And video can be ingested onto the XDN platform in other formats as well, including Real-Time Messaging Protocol (RTMP), Secure Reliable Transport (SRT), and MPEG-Transport Protocol (TS). The XDN retains these encapsulations while relying on RTP as the underlying real-time transport mechanism.

The XDN platform also provides full support for the multi-profile transcodes used with ABR streaming by utilizing intelligent Edge Node interactions with client devices to deliver content in the profiles appropriate to each user. And to ensure ubiquitous connectivity for every XDN use case, the platform supports content delivery in HTTP Live Streaming (HLS) mode as a fallback. In the rare instances where devices can’t be engaged via any of the other XDN-supported protocols, they will still be able to render the streamed content, albeit with the multi-second latencies that typify HTTP-based streaming.

XDN Nodes can be deployed on multiple cloud infrastructure-as-a-service (IaaS) platforms. This can be done by leveraging pre-integrations with major suppliers like AWS, Google Cloud, Microsoft Azure, and DigitalOcean or through integrations with many other IaaS platforms enabled by Red5 Pro’s use of the Terraform multi-cloud toolset.

The opportunity to make the smart TV a focal point for service enhancements tied to real-time audience engagement is at hand for pay TV and OTT service providers alike. To learn more about how the XDN platform can bring this opportunity to life, contact info@red5.net or schedule a call.