Ideally, we would be able to experience live events in-person. The coronavirus has changed all that. Although, even before the pandemic there were limitations to in-person events as well. Some fans may not have been able to afford a ticket, travel to a venue or the venue itself reached capacity. Having a way of broadcasting these events is the only way to extend that live experience to others. Because of this we strongly believe that new engaging live streaming experiences are here to stay even once we go back to normal.
This is why it’s even more important than ever to think hard about the design of your streaming infrastructure. Let’s take a look at some of the key features you should consider in choosing a platform to build a live event broadcasting application.
Live events are supposed to be exciting with large crowds cheering on their favorite artist or sports team. It’s the collective energy of the crowd that heightens the experience. In other words, it’s the interaction between the crowd and the performers, as well as within the crowd itself, that fuels the excitement.
Replicating this interactivity involves a few different elements. You can create in-app chat rooms which will expand the social aspect of event broadcasts. Fans can discuss specific topics with each other or share their reactions to in game events. These chat rooms can be created with a flexible API that allows you to create your own chat system or integrate with a third party chat provider. That same API could also allow you to create push notifications targeted to match a users’ interests, or an automatic signaling system that lets people know when someone replies to their comments. This will help them to engage with content and continue a dialogue, accelerating the pace of a conversation bringing the experience closer to how it would be in-person.
Expanding beyond chats there are ways to manipulate the video to increase visual interactivity as well. Creating video calls between friend groups will allow them to watch games together. It would also be possible to build a series of screens that could create a virtual display of fans. As this would be a large series of two way connections, the athletes or performers could hear the crowd and play off of their feedback. Perhaps with advancements in VR, friends will be able to gather virtually in the same room to replicate the experience of sitting together in the stadium and watching the game.
Incorporating multi-view capabilities further enhances the streaming experience. Viewers can switch from a wide angle shot of the field or stage to close-up shots of player reactions or the backup vocalists. Cameras set up from different sides of the venue capture the action from a variety of viewpoints that can only be virtually streamed.
Interactivity depends heavily upon the next two categories; latency and synchronization.
Live events happen in the moment so ensuring that viewers experience everything as it is happening is very important. How can you have a live event if it takes seconds to deliver the “live” event broadcast? You need a real-time latency.
America’s largest sporting event, The Superbowl this year experienced latencies of at least 28 seconds (that was the fastest!?) That’s plenty of time for someone at the stadium to text or tweet spoilers to one of the world’s most watched events, certainly no where near real-time streaming.
Solving this issue, requires a paradigm shift away from traditional CDN architecture and a movement towards protocols that were developed specifically for low latency. Unfortunately, the older protocols that most streaming platforms use are too slow, due to the fact that they are based on HTTP. Attempts to improve them such as Low Latency HLS, CMAF, Apple Low Latency HLS and even non-HTTP approaches like WebSockets and Media Source Extensions have all fallen short of delivering actual real-time latency. They deliver a latency measured in seconds which will never be fast enough for interactive experiences that consumers demand.
For browser-based applications, WebRTC is the only realistic choice to deliver sub 500ms of real-time latency. Native mobile apps can use RTSP to achieve the same performance. By delivering actual real-time latency, this helps improve synchronization as well.
As already mentioned above, spoilers are very annoying so preventing them is essential. However, there is more to synchronization than just that.
Ensuring that data matches what is streaming out is important as well. For example, game statistics delivered in a real-time graphic display should accurately reflect what is currently happening on the screen. You don’t want the score to be inaccurate causing confusion for the fans who might mistakenly think there was a penalty that prevented the point(s) from being awarded.
Additionally, synchronization is useful for sports betting as well. You want to ensure that accurate (and fair) bets will be placed at the right time. You also want to ensure that there isn’t a delay between the real-world action and when bets can be placed as that could allow for an unfair advantage with a person at the event texting the results to someone placing bets on the outside.
Beyond the fan experience, there is valuable user data that can be collected such as tracking which camera angles fans were watching or if they dropped off the feed at any point. The synchronization of that data needs to be accurate to be useful.
Also, synchronization is important for making sure that users can comment on the action as it happens rather than after. This helps maintain the exciting pace of live events.
Oftentimes, when broadcasting a live event the goal is to attract as many viewers as possible. This can only be done with proper scalability, otherwise users will be kicked out and have a negative experience.
Though this is intuitive, the process of actually implementing it can be rather complex. Not only do you need to scale to a large number of fans (especially for big headliners), but the fan experience should be the same for everyone on the stream. All subscribers should be able to join and see a real-time latency stream with the highest possible quality.
HTTP based networks that CDNs are built-on easily provide scalability, but at the great expense of latency, as discussed above. Additionally, CDNs work by caching data in different physical data centers which slows down the process of live streaming.
Cloud-based infrastructure is one way to get around the limitations of CDNs. By using cloud-based autoscaling, servers can be dynamically spun up or down as needed. Setting up autoscaling server clusters in different geographic regions provides global delivery wherever the fans are streaming from. Cross-cloud hosting solutions employ this approach as well unlocking a variety of locations depending upon the different cloud hosting providers.
Scalability is very important, but it needs to be effective as well. Running datacenters with gigantic servers isn’t worth the investment if your audience size is lower than expected, or even worse if the event is cancelled. That’s a lot of money wasted on extra CPU. As mentioned above, autoscaling will spin up servers on-demand to meet the current load.
Scaling back down is important as well. That will further reduce your server costs as fans drop off gradually during a lull in the action.
Furthermore, the software running the streaming platform must be efficient to maximize the number of connections per server. Dynamic scaling is great but not if you need to constantly spin up new servers each time a subscriber joins a stream.
Client side optimizations are important as well. For instance, ensuring that the streaming platform you are using supports codecs that feature hardware acceleration will be much more efficient at encoding and decoding. As opposed to software encoding and decoding which will be much slower. A good example of this is the more widely supported h.264 over H.65 and VP9.
6) Good Experience
Up to this point, we haven’t fully discussed one of the most important things from the fans’ perspective: How does the stream look and is it accessible?
A good live event broadcast should support a full range of resolution sizes from 4K down to 240p (and eventually up to 8K). Of course, not all fans will have access to the best or most reliable internet connections. The stream must flow smoothly with the highest possible quality.
Three features work together to provide the best possible experience: ABR, transcoding, and NACK.
Adaptive Bitrate (ABR) allows the client to request a lower bitrate that is more appropriate to the connectivity they are experiencing at that moment. That will ensure a smooth connection despite poor connectivity. Transcoding splits the stream into a variety of quality ladders (For example: high, medium, and low) so that users who can support the highest quality can subscribe to it, while users with poorer connections can still watch.
WebRTC (already recommended for real-time latency) provides quality assurances as well. Itself. Unlike slower protocols, WebRTC responds to stream conditions in real-time. As such, REMB messages sent over RTCP (an element of WebRTC used for sending flow control messages) allow the edge node to deliver just the right size stream for every network situation.
WebRTC also features NACK, which is a method of resending critical packets. A bad network connection will likely result in the client dropping packets. Rather than trying to resend each and every one of the packets, NACK identifies the ones that are most important and resends those. This prevents the network from getting further clogged with redundant requests. This will help keep the stream flowing and looking good even under poor network conditions.
7) Forward Thinking Platform
How you decide to build your live event broadcasting platform should factor future requirements as well. Building your application with forward thinking software is essential.
Tech is constantly evolving, with new standards emerging as older ones retire. Flash is the
most recent example of a once dominant piece of technology that at the end of this year will be no longer supported. For live streaming, HLS is next.
However, that came as no surprise to those following the industry and the development of the latest web-standard WebRTC. As it is supported by Apple, Google, Microsoft, Mozilla, and Opera, the WebRTC standard will remain up to date and functional for the foreseeable future.
Obviously, a well functioning platform is a very high priority. However, one should always be looking to the future, or use a software platform that is doing that for you. Just as Flash was the once dominant method of live stream delivery, HLS is the next piece of aging technology looking to be phased out.
8) Multi-Platform Support
Though you may not be able to see them in-person, you need to “meet” them where they are and support how they will watch the event. Part of that is covered through geographic distribution as discussed above but what about all those devices? From a myriad of laptops with different browsers, smartphones, as well as tablets, fans will want to watch the event with a variety of devices.
WebRTC can run directly in all the major browsers (without a plugin) including Chrome, Safari, Firefox and Edge. For mobile support, there are mobile browsers of course, (also supported by WebRTC) but that’s not the most effective solution. A mobile SDK is the best option to ensure consistent performance. Support for IoT devices such as VR headsets should be considered as well.
Failover support to RTMP and HLS ensures compatibility with older devices.
Of course, there is more than just watching the streams. There is broadcasting to consider as well. Drones will help ensure that a variety of camera angles are captured and incorporated into the broadcast.
As hinted throughout this post, customization is the keystone of any effective live event broadcasting platform. Security considerations, multi-view cameras, collecting information on viewing habits, not to mention controlling your own back-end infrastructure (more on that below) all depend upon a flexible software platform.
Custom features are the best way to make your app stand out. For example, fans might access on-demand data such as player info from statistics and integrated heartbeat monitors. This will further increase their engagement.
Of course, depending upon your business model, there is also the consideration of ad insertions.
Fans may want to keep a recording of a show or important game (even if it’s just deleted in anger when they lose). This could be expanded to include replays or other personal recording options. Again, this is where synchronization would be very important to ensure proper time sequencing.
11) Flexible Off-Site Production
Especially in the face of covid restrictions, moving operations to partially or fully of-site production spaces is more important than ever. Whether due to pandemic restrictions or avoiding travel costs, being able to work together when you can’t all be in the same control room can be difficult. Latency and cloud-based hosting are the keys for ensuring natural conversation flows, being able to react to events happening in real-time and to connect with other people on the production team.
Furthermore, using home-base production facilities to process video flows from the field without having to send big production vans or pay for costly, latency-burdened satellite uplinks introduces new levels of versatility with faster turnaround as well as significant cost savings.
Remote collaboration enables the development of new types of content, extra features and interactive applications, most of which are already covered in this post. This is due to the fact that effective off-site production will capture the raw A/V feeds, including on-site announcers’ commentary.
Protection from illegal streaming and other hackers is very important. Forensic watermarking is an effective strategy at tracking down the source of a pirated stream. Yet again, latency plays an important role here as the need to respond quickly is very important.
To prevent hackers, it is important to encrypt your streams. Encryption is the process by which information is encoded into a format that makes it unrecognizable until it is decoded with a key. In this way, information can be securely sent over the open internet from a client to a server. Once it reaches the server it must be decrypted in order to be rerouted to other pipelines so it can eventually be read and understood.
While Red5 Pro uses WebRTC which encrypts all streams, what can be done about unauthorized users? Using Red5 Pro’s round trip authentication system, enables stream authentication before gaining access to the streams. This allows you to fully customize how access is given to users.
Where and how you host your live event platform is important. The convenience of choosing to use Platform as a Service (PaaS), can have disastrous consequences if you end up in a service trap. As Platform as a Service (PaaS) providers make decisions, you may need to move your application onto one that better supports your needs.
PaaS providers are subject to the same issues as any other business. A buyout from another company, changing market trends, or just internal decisions can trigger changes. Those changes could be dropping a region where many fans live, or a drastic change in their pricing. You don’t want to be locked into working with a company that no longer works well with your application through fear of having to rebuild your entire application.
The best option is then to use a hosting agnostic solution with a flexible API to ensure you can have the freedom to port your application over to another hosting provider. Multi-platform support eliminates the risk of being permanently fixed to a single provider. The aforementioned mobile SDKs should be portable as well as you don’t want to tear everything down and have to build it all up again.
14) Technical Support
This may be a little counterintuitive as an argument could be made that a well built platform won’t need support. However, most people would agree that even the most well built products will need the occasional technical support.
With customizable platforms in particular, it is essential that you receive guidance in making the most out of all the available features. Chat channels, online ticketing systems, and advanced support contracts can all be leveraged to make sure that all your needs are met.
Covid has greatly affected our daily lives and accelerated efforts already in progress regarding live streaming. This is especially true for sports, theatre and music venues struggling to safely open up and increase their viewership. The above features will greatly aid in building a live streaming application that could effectively bring us “back” to these events and in some ways even accentuate them.
Interested in building your own live event broadcasting platform? Let us show you what Red5 Pro can do by sending an email to firstname.lastname@example.org or schedule a call.