11 Things to Consider when Building an iOS Live Streaming App

Building an iOS live streaming app can be complicated and tedious. On top of all the backend system infrastructure required for any web application there are additional considerations for native app integration. As such, some live streaming software companies offer mobile SDKs to help make this process easier.

Let’s take a look at some of the key features you should consider in choosing a platform and SDK to consider when building an iOS live streaming app.

1) Latency

It can only be considered live if your viewers can see what is happening as it is actually happening. If it takes more than half a second for the video to travel back and forth from the publisher and the subscriber the whole experience will feel unnatural and slow.

Conference apps, event broadcasts, live auctions, drone guidance, or any kind of conversation all need real-time latency. Even a small delay of just seconds between the broadcaster and subscriber results in a negative user experience. High latency creates stuttered conversations, spoilers and a basic sense that something is off and the live event doesn’t actually feel live.

Achieving real-time latency in your iOS live streaming app requires the use of protocols that have been developed specifically for low latency. When it comes to iOS applications, RTSP is well suited for the fast and reliable delivery of live streams. Like WebRTC, RTSP delivers a stream with under 500 ms of latency. This is due to the fact that RTSP uses RTP (or SRTP for a secure connection) which is the same transport protocol that WebRTC uses. That is one of the reasons why we at Red5 Pro chose to leverage RTSP in our mobile SDKs.

Other approaches to lower video latency including Low Latency HLS, CMAF, Apple Low Latency HLS and WebSockets and Media Source Extensions have all fallen short of delivering actual real-time latency. RTMP is also another common protocol used in mobile streaming apps, but it is limited as well and typically adds more latency to your app’s video experience.

For more on how RTSP provides low latency, take a look at this blog post.

2) Scalability

Having low latency is important. Maintaining that low latency for all your users is equally important, especially when you are thinking of scaling to millions of concurrent users.

In order to scale, some streaming platforms like Vonage Video API (formerly TokBox) switch to higher latency protocols after reaching a certain threshold (currently 3,000). This means that any additional viewers joining after the switch, have a different experience from those that joined earlier.  While this allows for the stream to be delivered to more people, the tradeoff of high latency and a poor user experience is a steep price to pay.

Even worse, some platforms are built entirely on traditional CDN infrastructure that use high latency, HTTP based protocols by default. Under those conditions, no users will get low latency at all. Wowza for example explains that CDNs are critical for any live streaming app built on their platform, which translates to high latency.

When considering the goal of providing the best user experience, the challenge lies in scaling out to large audiences while maintaining real-time latency.

One way to address this is by leveraging cloud networks to spin up new compute instances to create a cluster of server nodes which deliver content without relying on the caching mechanisms that CDNs use. Using this approach allows the platform to scale to millions of concurrent users while maintaining sub 500 milliseconds of latency. This RTSP based delivery design creates an opportunity to broadcast large sports events, concerts or other events making a much more interactive experience possible.

For more in depth information on how we approach scaling on cloud networks, take a look at this post on our cross cloud distribution system.

3) Efficiency

Scalability is great, but without efficiency, your users could be left waiting around while the back end infrastructure struggles to setup everything appropriately. Scaling out and maintaining low latency is great but you also need a streamlined platform that maximizes the number of connections you can get per server instance.

Of course, you will also need a stable connection ensuring that clients and broadcasters can continue to participate in a live stream without any interruptions. The connection logic needs to work well. Additionally, features such as the use of relay nodes will help optimize streaming performance while allowing for scale.

An efficient media server implementation will also result in more streams per server instance which further streamlines data delivery. In addition the more efficient the protocols used, the more streams each server instance can handle. This avoids the need to run a massive fleet of cloud instances to support large events.

One such example of this kind of protocol optimization, is Red5 Pro’s use of RTSP for their mobile SDK. When compared to browser-based WebRTC streaming, RTSP is lightweight, which enables you to get 5 times more streams per server instance.

More specifically to iOS, a good streaming SDK needs to leverage hardware encoding and decoding as well. Apple devices ship with highly efficient chipsets for decoding and encoding live video and audio.  Many solutions try to accomplish the encoding and decoding using pure software but doing so can drive up power consumption, peg the CPU and drain your battery quickly. Luckily Apple makes accessing their hardware video chipsets fairly easy through VideoToolBox. Thus the most effective solutions do not use pure software for encoding and decoding the live video stream.

Graphic acceleration is also important for rendering and display graphics. Your app should leverage the latest Metal API or OpenGL for rendering your video. If you’re not taking advantage of efficient graphic rendering on iOS, displaying live video will kill the battery. Being forced to charge your phone means tethering to a wall outlet: defeating the purpose of a mobile device. This makes the use of hardware acceleration very important, whether it’s an encoder chip, decoder, or graphics card.

If you really want to get sophisticated, take a look at our example of using Metal, or take it even further by creating your own 360º video render.

4) Portability

Surviving (and hopefully thriving) within the tech sector requires agility. There is a need to address changes within the structure of your application as you respond to evolving customer expectations.

Platforms you used to build your application can change from an external merger that changes pricing, to internal company decisions dropping features, or outright going out of business. What can make or break an application is the ability to proactively respond to these changes, as being locked-in to a particular platform can have harsh consequences.

With this in mind, a hosting agnostic solution with a flexible API is preferable in order to maintain flexibility. By supporting multiple platforms, you can avoid being forcefully tied to a single provider. AWS, GCP, Azure, and DigitalOcean are just some of the few hosting providers with a variety of disadvantages and benefits. With an API that allows you to plug into any back-end, you can set up your application on different systems without resorting to overhauling the entire infrastructure.

Of course you will want to make sure that your solution provider is in it for the long run like we are at Red5 Pro. Some groups are looking to make a product and company to be sold quickly leaving their users in a lurch when that acquisition happens.

5) ABR

Not all users will have perfect, 5G connectivity. Even those in highly connected areas can have periods of throttled bandwidth or instability.

To ensure a smooth video delivery to every customer possible, responsive features such as Adaptive Bitrate (ABR) are crucial. ABR works by allowing the client and/or publisher to automatically request a lower bitrate more appropriate to the connectivity they are currently experiencing. Having the ability to switch bitrates for both the broadcast and subscribing side of the stream, maximizes its effectiveness.

As compared to high latency HTTP based protocols, the approach to implement ABR is quite different when dealing with real-time protocols like RTSP. You can’t rely on simply requesting a new manifest and pulling new segment files as is done with MPEG DASH and HLS. In order to adjust on the fly with RTSP the system needs to be able to shift to the different stream variants via information provided over a control protocol.

More information on our approach to ABR and transcoding streams is detailed in the Red5 Pro server documentation with a corresponding example for iOS on Github.

6) Backwards Compatibility

Most users (especially iOS users) tend to upgrade to the latest OS fairly quickly. However, that does not mean that all phones can actually support the latest OS so you want a platform that can support backwards compatibility. Red5 Pro specifically supports iOS version 9.2+. This gives your users enough time to be able to upgrade at their own pace rather than forcing them to keep up with the rapid update cycle.

However, support for legacy OS is not the only thing you need for compatibility.

7) Compatibility with Other Devices

Your iOS app would be quite limited if it could only communicate with other iOS devices. Supporting a variety of platforms and hardware makes it much more valuable and useful. What use is a highly scalable platform if it only supports limited kinds of devices?

Considering that you’re building an iOS app, you also likely want to release an Android app. Of course, your users might want to use a laptop or desktop as well so that involves supporting a variety of browsers with near real-time video latencies like Red5 Pro’s HTML5 SDK. Ensuring that you won’t be restricted to a single platform is essential for increasing your reach.

Beyond just mobile devices, you might want to support IoT devices as well. IP cameras, Drones, and VR headsets can all enhance the live video streaming experience.

The Red5 Pro solution can transcode from RTSP to WebRTC which can be run directly in the browser without a plugin. WebRTC has the support of the major tech players, so it can be run in Chrome, Firefox, Safari, Edge, and Opera. You can use mobile as well as desktop browsers. You can also tap into streams from IP Cameras and other devices running SRT, RTSP or MPEG-TS using our restreamer plugin.

These flexible options make sure that you can stream to all the devices your customers are looking for.

8) Customizable

“One size fits all” hardly ever does. It might work for someone else, but that doesn’t mean it will work for you.

We are all familiar with devices that offer a variety of features many of which are never used. Why have all these extras bogging down the system? Furthermore, you may want to setup custom security features, special UI interfaces, or additional features. If all you have to choose from is a list of standard features, how else can you stand out?

The Red5 Pro iOS SDK itself has a lot of flexibility allowing you to pull in custom video sources and audio sources, manipulate the streams on device before they go out to your users. You can also access the raw frames of the audio and video on the subscriber side to do things like chroma key greenscreen effects and more . These features and this level of customization can be critical in building a custom iOS live streaming app.

However, customization is only valuable if you actually implement working features which brings us to our next point:

9) Technical Support

While some may argue that if a platform is well built you shouldn’t have to worry about support, that’s just not true. Whether you’d like to add something completely new, ensure that you have optimized everything as effectively as possible, or just need someone to point out the sentence you missed in the documentation (it happens to all of us), you will likely need support at some point.

The more flexibility and options a platform offers, the more necessary good support becomes. What is the point of customization if no one shows you the best way to do it? A custom suit needs a tailor, afterall. You need a good support system with chat channels, online ticketing systems, and advanced support contracts to help meet all your needs.

Some companies treat a mobile SDK as an afterthought and their support team can reflect this. If you are looking for technical support for a mobile SDK you want to make sure that the person answering your question or addressing your issue has developer experience in building mobile apps.  Sure the back-end server side infrastructure of an application is important but it shouldn’t come at the expense of everything else.

10) Forward Thinking Platform

The highest priority is a well functioning platform. However, it should also continue to perform well. As the recent downfall of Flash has demonstrated, the technologies that drive live streaming are subject to change.

New standards are always being developed. More efficient codecs and streamlined protocols are all part of the marketplace. Of course, “new” doesn’t always mean “better”, so you also want to partner with an organization that can prioritize the correct improvements.

Rather than creating a high performance video streaming platform and stopping, a good company continues to perfect their product. They make adjustments, respond to customer feedback and generally work to improve everything.

11) Well Tested and Proven

It goes without saying that a well designed platform is only as good as it’s implementation, and will only work well when tested from all sides. Extensive testing and consistent performance are important to your application.

With automatic cloud based scaling, sub 500 milliseconds of latency, cross device compatibility, and a fully customizable API, Red5 Pro is trusted by major corporations such as the New York Times, Sony, Novetta, Invaluable Auctions, Google, and many others. Furthermore, several US government agencies use Red5 Pro in some of their command control, and drone surveillance systems.

It should also be noted that building live real-time video streaming for iOS from scratch can be hard. Live streaming in and of itself is challenging which is why we built Red5 Pro in the first place, but adding a mobile SDK is a whole additional layer. Why waste your time pouring developer hours and money creating something that already exists?

Looking to build a new application or switch over an existing one? See what Red5 Pro can do for you. Send an email to info@red5.net or schedule a call directly. Happy coding!