12 Things to Consider When Building a Live Streaming App

Building a live streaming application requires many moving pieces. Not only is the process of live streaming complicated, but the number of companies offering to help “simplify” the process can have the opposite effect.

Let’s take a look at some of the key features you should consider in choosing a platform to build your live streaming application on.

1) Latency

Having the lowest possible latency is a keystone to live experiences.

Live auctions, drone guidance, event broadcasts, even basic conversations all need real-time latency. Any sort of delay between the broadcaster and subscriber will have negative consequences. High latency results in staggered conversations, spoilers and a general sense that the live event is not actually live.

In order to achieve real-time latency, you must use protocols that have been developed specifically for low latency. For browser-based apps WebRTC is really your only choice to get near real-time latency. WebRTC when handled correctly consistently achieves under 500 milliseconds of latency. That is fast enough to be nearly imperceptible.

Other efforts to lower browser based video latency including Low Latency HLS, CMAF, Apple Low Latency HLS  and WebSockets and Media Source Extensions have all fallen short of delivering actual real-time latency.

For more on how WebRTC provides such low latency, check out this blog post.

If it’s a native mobile app you are building you have a few more options. RTSP is a very good protocol for mobile because the underlying video transport is RTP or its secure cousin SRTP, which also happens to be exactly what WebRTC uses. RTSP also happens to be a much simpler protocol than WebRTC better geared towards a client server model. Thus RTSP typically allows for more connections on your server endpoint making your server infrastructure scale with lower costs.

While the rest of this post talks mostly about streaming solutions based on WebRTC, the same exact principles apply to RTSP based video streaming apps.

2) Scalability

Equally important to having a low latency stream is the ability to send that out to as many people as needed. Importantly, it needs to maintain that low latency.

Some streaming platforms like Vonage Video API (formerly TokBox) switch to higher latency protocols after they meet a certain threshold, meaning that those additional viewers won’t have the same experience as those that joined earlier. While that allows them to achieve scalability, it comes with the tradeoff of high latency and a poor user experience.

However, some platforms based on traditional CDN infrastructure use HTTP based high latency protocols by default. That means none of their users get low latency at all.

In order to provide the best user experience, the challenge lies in scaling out to large audiences while maintaining real-time latency for the best user experience.

One way to solve this is by leveraging cloud networks to spin up new compute instances to create a cluster of server nodes that deliver content without the caching mechanisms that CDNs rely on. Using this approach the platform can scale to millions of concurrent users while maintaining sub 500 milliseconds of latency. This WebRTC based delivery design creates an opportunity to broadcast large sports events, concerts or other events making a much more interactive experience possible.

3) Efficiency

Your live streaming app needs to work efficiently so your users are not waiting around for the back end infrastructure to setup everything appropriately. Scaling out is great but without streamlining the CPU consumption of the live streams your network could be bloated with many unnecessary servers.

There also needs to be stable connection logic to ensure that clients and broadcasters are connected is a way that works well. Additional features such as relays can help optimize streaming performance as well.

Furthermore, efficient coding will result in compact streams which will streamline data delivery. This enables more streams per server and reduces the need to run hordes of servers to support large events with lots of connections.

4) Portability

Thriving within the tech sector requires agility. Beyond responding to evolving customer expectations, there is a need to address changes within the structure of your application.

Platforms that you used to build your application can change from internal company decisions to drop features, an external merger that changes pricing, or outright going out of business. The ability to proactively respond to these changes can make or break an application, as being locked-in to a particular platform can have dire consequences.

A hosting agnostic solution with a flexible API is preferable in order to maintain flexibility. That way multiple platforms will be supported to avoid being locked-in to a single provider. AWS, GCP, Azure, and DigitalOcean are just some of the few providers with a variety of benefits and disadvantages. An API that allows you to plug into any back-end, allows you to setup your application on different systems without having to overhaul the entire infrastructure.

5) ABR

Not all users will have perfect, 5G connectivity. Even those in highly connected areas can have periods of instability or throttled bandwidth.

In order to ensure the smooth delivery of video to every customer possible, responsive features such as Adaptive Bitrate (ABR) are very important. ABR allows the client to request a lower bitrate that is more appropriate to the connectivity they are experiencing at that moment.

The approach to implement ABR is quite different when dealing with real-time protocols like WebRTC. You can’t rely on simply requesting a new manifest and pulling new segment files as is done with MPEG DASH and HLS. In order to adjust on the fly with WebRTC the system needs to be able to shift to the different stream variants via information provided over a control protocol. REMB messages sent over RTCP allow the edge node to deliver just the right size stream for every network situation.

6) Modern Protocols

Keeping up to date with the latest technology is quite obviously important to a tech platform. As the phasing out of Flash has shown, well established protocols that were once dominant, can go away.

WebRTC has emerged to take the place of Flash (and RTMP) as a widely supported and effective browser based protocol. Capable of transporting live streaming video in near real-time, WebRTC is suited for the modern demands placed upon live streams.

Based on web-standards, WebRTC also enjoys the support of Apple, Google, Microsoft, Mozilla, and Opera. That support ensures that the WebRTC standard remains up to date and functional for the foreseeable future.

That being said, you also need…

7) Support for Legacy Systems

Even though Flash is going away, RTMP (the capable workhorse of live streaming) will still be around. As such, you will need to have RTMP to support older browsers/devices. Look for solutions with failover support to RTMP and HLS to ensure that your streams will reach all your users.

However, support for legacy protocols is not the only thing you need for compatibility.

8) Multi-Platform Support

Your video streaming server will need to work across a variety of different devices and browsers. You wouldn’t want to alienate any of your potential customers by not supporting their device of choice. Android users are painfully aware of this whenever they hear about a cool new app that’s only available in the Apple Store.

WebRTC can be run directly in the browser without a plugin. As mentioned earlier, it has the support of the major tech players, so it can be run in Chrome, Firefox, Safari, Edge, and Opera. More importantly, you can use mobile as well as desktop browsers.

Furthermore, you should also have the flexibility of adding a native app with a mobile SDK. The ability to stream in mobile browsers or in an app, allows you to support mobile devices even if you don’t have the resources to devote to customizing a native app.

Beyond just mobile devices, you might want to support IoT devices as well. Drones, IP cameras, and VR headsets can all benefit from live video streaming. These flexible options ensure that you can stream to all the devices your customers are using.

9) Technical Support

This may be a little counterintuitive as an argument could be made that a well built platform won’t need support. However, most people would agree that even the most well built products will need the occasional technical support.

The more flexibility and extensible features a platform offers the more having good support becomes a necessity. With customizable platforms it is essential that you receive guidance in making the most out of all the available features. Chat channels, online ticketing systems, and advanced support contracts can all be leveraged to make sure that all your needs are met.

10) Customizable

“One size fits all” hardly ever does. Just because it worked for someone else, doesn’t mean it will work for you.

Those that just bought a new phone are all too familiar with this as they struggle to eliminate the bloatware pre-installed (sometimes permanently) on their new device. If you don’t need something why should you be forced to have it?

Conversely, there is the issue of needing something and not being able to get it. Whether it’s a security configuration or special feature, you need a solution that lets you build what you want with no restrictions.

11) Forward Thinking Platform

Obviously, a well functioning platform is a very high priority. However, another major consideration should be how well it will continue to perform. As the recent downfall of Flash has shown, the technologies that drive live streaming can be subject to change.

There are always new standards in development. Streamlined protocols and more efficient codecs are all part of the marketplace.

A good video streaming platform doesn’t just create a product and stop. They make adjustments, respond to customer feedback and generally work to improve everything.

12) Well Tested and Proven

There are no amount of features that will make up for a poor performing streaming platform. Extensive testing and consistent performance are important to your application.

With WebRTC support creating latency under 500 milliseconds, automatic cloud based scaling, a fully customizable API, and cross device compatibility, Red5 Pro is trusted by major corporations such as Google, the New York Times, Sony, Novetta, Invaluable Auctions, and many others. Even more telling is the fact that the US military uses Red5 Pro in some of their drone surveillance and command control systems.

Looking to build a new application or switch over an existing one? See what Red5 Pro can do for you. Send an email to info@red5.net or schedule a call directly. Happy coding!