Red5 Pro Autoscaling - Transcoding and ABR

Transcoding allows a user to publish multiple variants of a single video stream, and deliver to the subscribers the best variant given their available bandwidth. In particular, Red5 Pro server-side transcoding allows for:

  • Publishing multiple provisioned streams.
  • Publishing a single stream to a transcoder within Red5 Pro that generates provisioned streams.
  • Subscribing to a stream via adaptive bitrate, which will allow dynamic upgrading and downgrading of the stream quality based on network conditions.

Figure 1 shows an overview of the system when different variants of the same stream are published using a media encoder. The process works as follows:

  1. A provision is provided to the stream manager to specify the different variants of the stream that will be published.
  2. The stream manager selects a suitable origin and provisions it to expect multiple versions of the same stream.
  3. The stream manager returns a JSON object with the details of the origin endpoint that should be used to broadcast.
  4. The broadcaster can then use its media encoder to publish to the origin.

Figure 1: Overview of the system when a media encoder is used to publish multiple versions of a stream.

Figure 2 shows an overview of the system when using a transcoder node. The process works as follows:

  1. A provision is provided to the Stream Manager. It specifies the different variants of the stream that will be published. Then, the Stream Manager API is used to request a transcoder endpoint.
  2. The Stream Manager selects a suitable origin and transcoder. The origin is provisioned to expect multiple versions of the same stream while the transcoder is provisioned with the details of the different stream variants that it needs to create.
  3. The Stream Manager returns a JSON object with the details of the transcoder endpoint that should be used to broadcast.
  4. The broadcaster can then start to publish a single stream to the transcoder.

Figure 2: Overview of the system when a transcoder server is used to generate multiple versions of the same stream.

This document will explain the process that needs to be followed and the API calls involved to deliver an adaptive bitrate video stream.

Requirements

Adaptive Bitrate streaming is supported by:

  • Red5 Pro Server v5.0.0 or higher
  • Red5 Pro HTML SDK v5.0.0 or higher

Additionally, the use of the Red5 Pro Autoscaling solution and Stream Manager API is required for an Adaptive Bitrate subscriber experience.

Note: Adaptive Bitrate subscribing is not yet supported by the iOS and Android SDKs.

Provisioning

The first step to set up an Adaptive Bitrate video streaming is to provide a provision of stream variants to the Stream Manager. This is achieved by making a POST request with a JSON object as payload:

Method:POST

URL: https://yourstreammanager.com/streammanager/api/3.1/admin/event/meta/live/mystream?accessToken=myaccessToken

Where:

  • yourstreammanager.com: is the url of your Stream Manager
  • mystream: is the top-level name (guid) of the stream to be published
  • accessToken: is the access token defined in the Stream Manager configuration.

The JSON object needs to have the following structure:

{
  "meta": {
      "authentication": {
      "username": "<username>",
      "password": "<password>"
     },
      "stream": [
        {
          "name": "<variant1_name>",
          "bandwidth": "<bytes>",
          "level": "1",
          "properties": {
          "videoHeight": <value>,
          "videoWidth": <value>,
          "videoFPS": <value>
          }
         },
                 {
          "name": "<variant2_name>",
          "bandwidth": "<bytes>",
          "level": "2",
          "properties": {
          "videoHeight": <value>,
          "videoWidth": <value>,
          "videoFPS": <value>

          }
         },
                 {
          "name": "<variant3_name>",
          "bandwidth": "<bytes>",
          "level": "3",
          "properties": {
          "videoHeight": <value>,
          "videoWidth": <value>,
          "videoFPS": <value>

          }
         }
],
      "georules": {
      "regions": [
        "<location1, location2. etc>"
      ],
      "restricted": "<restriction_status>"
    },
    "qos": "3"
  }
}

Where:

  • Stream: an array that defines, for each version of the same stream, an object with the stream’s level, name, bitrate, height and width, and frames per second (optional).
  • Authentication (optional): specifies the username and password that have been configured for the video stream if required.
  • Qos (optional): is an integer that indicates the quality of service that needs to be provided (currently 3 is the only valid level).
  • Georules (optional): based on the restricted boolean value, this specifies which regions should be allowed or restricted to subscribe to a video stream.

As an example, using mystream as the top-level name and defining three variants for the stream, the POST and JSON object would look like the following:

https://yourstreammanager.com/streammanager/api/3.1/admin/event/meta/live/mystream?accessToken=myaccessToken

{
  "meta": {
      "authentication": {
      "username": "",
      "password": ""
     },
      "stream": [
        {
          "name": "mystream_1",
          "bandwidth": "1000000",
          "level": "1",
          "properties": {
          "videoHeight": 1280,
          "videoWidth": 720,
          "videoFPS": 24
          }
         },
                 {
          "name": "mystream_2",
          "bandwidth": "750000",
          "level": "2",
          "properties": {
          "videoHeight": 640,
          "videoWidth": 360,
          "videoFPS": 24

          }
         },
                 {
          "name": "mystream_3",
          "bandwidth": "300000",
          "level": "3",
          "properties": {
          "videoHeight": 320,
          "videoWidth": 180,
          "videoFPS": 15

          }
         }
],
      "georules": {
      "regions": [
        "US"
      ],
      "restricted": "false"
    },
    "qos": "3"
  }
}

A successful response will return an object with the top-level stream name and the meta field populated with the JSON object sent in the POST call:

{
    "name": "mystream",
    "scope": "live",
    "data": {
        "meta": {
            "authentication": {
                "username": "",
                "password": ""
            },
            "stream": [
                {
                    "name": "mystream_1",
                    "bandwidth": "10000000",
                    "level": "1",
                    "properties": {
                        "videoWidth": 1280,
                        "videoHeight": 720,
                        "videoFPS": 24
                    }
                },
                {
                    "name": "mystream_2",
                    "bandwidth": "800000",
                    "level": "2",
                    "properties": {
                        "videoWidth": 640,
                        "videoHeight": 360,
                        "videoFPS": 24
                    }
                },
                {
                    "name": "mystream_3",
                    "bandwidth": "350000",
                    "level": "3",
                    "properties": {
                        "videoWidth": 320,
                        "videoHeight": 180,
                        "videoFPS": 24
                    }
                }
            ],
            "georules": {
                "regions": [
                    "US"
                ],
                "restricted": "false"
            },
            "qos": "3"
        }
    },
    "updated": 1539880127208
}

It is possible to access the provision at any time through the Stream Manager API as follows:

Method:GET

URL: https://yourstreammanager.com/streammanager/api/3.1/admin/event/meta/live/mystream?accessToken=myaccessToken

Where:

  • yourstreammanager.com: is the host of your Stream Manager
  • mystream: is the GUID of the stream
  • accessToken: is the token defined in the Stream Manager configuration.

Publishing

Once the provision has been provided to the Stream Manager it is possible to start publishing. It is possible to publish in two different ways:

  • Using an RTMP or RTSP media encoder to publish the different variants of the same stream.
  • Publishing the high level stream variant via WebRTC, RTMP or RTSP to a transcoder node to publish a single high quality version and create all the other variants.

Publishing with an encoder

Once the provision has been provided to the Stream Manager it is possible to start publishing. Make a broadcast request to the stream manager to get the IP address of the most available origin server. The provision specifies the different variants of the video stream that need to be provided. Each of these variants can be broadcasted with a media encoder like Wirecast, Flash Live Media Encoder or others. The different entities involved and their connections are shown in Figure 3. In this case the same origin is used for all the different variants. The origin then sends the streams to the relays which then deliver it to the edges that eventually serve the subscribers.

Figure 3: Overview of the system when a media encoder is used to publish three variants of a stream.

For each stream variant the following call needs to be made:

Method:GET

URL: https://yourstreammanager.com/streammanager/api/3.1/event/live/mystream_1?action=broadcast

Where:

  • yourstreammanager.com: is the host of your Stream Manager
  • mystream_1: is the name of the stream variant that will be published

If the call is successful the Stream Manager will return an object that specifies the scope and address of the origin to use. These properties will then be used to define the endpoint in the media encoder stream configuration.

As an example, if the JSON object returned by the Stream Manager is:

{
     "serverAddress": 127.0.0.1,
     "scope": "live",
     "name": "mystream_1"
}

You would want to broadcast, using RTMP, to:

rtmp://127.0.0.1:1935/live/mystream_1

Publishing with a transcoder

Publishing with a transcoder allows to publish a single video stream that should be in high quality and use the transcoder to generate the other variants of the stream at lower quality. These variants are then broadcasted to an origin which will then send them to a set of relays. The relays will deliver the variants to the edges which will use REMB to determine what variant to serve to each subscriber based on the conditions of the network. An overview of the system and the connections between the servers is given in Figure 4 where, for simplicity, a single origin, relay edge, and transcoder are used.

Figure 4: Overview of the system and the server connections when publishing a single stream with a transcoder.

Once the provision has been provided to the Stream Manager it is necessary to make a request to retrieve a transcoder endpoint. The request is as follows:

Method:GET

URL: https://yourstreammanager.com/streammanager/api/3.1/event/live/mystream?action=broadcast&transcode=true

Where:

  • yourstreammanager.com: It is the host of your Stream Manager
  • mystream: It is the top-level name of the stream that needs to be published

A successful response is a JSON object that specifies the serverAddress, scope and name that need to be used to start publishing.

As an example, the response JSON object will look like this:

{
     "serverAddress": 127.0.0.1,
     "scope": "live",
     "name": "mystream"
}

Publishing to a transcoder with WebRTC

When publishing with WebRTC it is necessary to use the Stream Manager proxy. The proxy will transfer the publish request to the transcoder provided. This is necessary because browsers require that WebRTC broadcasts are served over SSL. Moreover, in an autoscale environment in which Red5 Pro instances are spun up and down dynamically based on load, it can be unfeasible to have certs on each of them.

As an example, if the Stream Manager response to a request to publish using a transcoder is:

{
  "serverAddress": "10.0.0.0",
  "scope": "live",
  "name": "mystream"
}

Then the initialization configuration for a RTCPublisher will look like the following:

(function (red5prosdk) {

  var publisher = new red5prosdk.RTCPublisher()
  publisher.init({
    host: 'yourstreammanager.com',
    app: 'streammanager',
    streamName: 'mystream',
    protocol: 'wss',
    port: 8083,
    mediaConstraints: {
      audio: true,
      video: {
        width: 1280,
        height: 780
      }
    },
    bandwidth: {
      video: 1000
    },
    connectionParams: {
      host: '10.0.0.0',
      app: 'live'
    }
  })
  .then(function () {
    publisher.publish()
  })
  .catch(function (e) {
    console.error(e)
  })

})(window.red5prosdk)

Where it is necessary to assign the top-level app property as streammanager and provide a connectionParams object that details the endpoint to proxy to.

Publishing to a transcoder with Flash

When publishing with Flash it is not necessary to use the Stream Manager proxy. That is because there is no requirement to publish over SSL from the Flash Player plugin. Assuming that the object received from the Stream Manager is the same as the one used in the previous example, then the initialization configuration for a RTMPPublisher will look like the following:

(function (red5prosdk) {

  var publisher = new red5prosdk.RTMPPublisher()
  publisher.init({
    host: '10.0.0.0',
    app: 'live',
    streamName: 'mystream_1',
    protocol: 'rtmp',
    port: 1935,
    mediaConstraints: {
      audio: true,
      video: {
        width: 1280,
        height: 780
        bandwidth: 1000
      }
    }
  })
  .then(function () {
    publisher.publish()
  })
  .catch(function (e) {
    console.error(e)
  })

})(window.red5prosdk)

Subscribing

With a provision available from the Stream Manager and multiple variant broadcasts being streamed (either through using a media encoder or the transcoder) it is possible to subscribe to an adaptive bitrate stream. When a client subscribes it receives the stream variant with the highest quality given its network conditions. This is achieved by having the edge interact with the subscriber client. In particular, the edge and subscriber will use a RTCP message called Receiver Estimated Maximum Bitrate (REMB) to provide a bandwidth estimation. Then, the edge will use the bandwidth estimation to make the corresponding adjustments to video quality. In particular, the edge will deliver the video with the highest quality given the estimated bandwidth.

Subscribing with WebRTC

When subscribing with WebRTC it is necessary to use the Stream Manager proxy which will transfer the subscribe request to the edge provided. This is necessary because browsers require that WebRTC uses SSL connections. The proxy avoids to have to deploy a cert on every edge which would be unfeasible when using an autoscale environment, where Red5 Pro instances are spun up dynamically based on load.

The first step to subscribe with WebRTC is to request an edge from the Stream Manger. The request needs to specify the stream name of the variant to which you want to subscribe to. It is recommended to first subscribe to a variant that is in mid-range and then let the edge make any adjustments if necessary. The request to the Stream Manager is as follows:

Method:GET

URL: https://yourstreammanager.com/streammanager/api/3.1/event/live/mystream_2?action=subscribe

Where:

  • yourstreammanager.com: is the host of your Stream Manager.
  • mystream_2: is the name of the stream variant that you want to subscribe to.
  • accessToken: is the token defined in the Stream Manager configuration.

A successful response from the Stream Manager will look like this:

{
  "serverAddress": "10.0.0.0",
  "scope": "live",
  "name": "mystream_2"
}

This data is then used for the initialization configuration for a RTCSubscriber which will look like the following:

(function (red5prosdk) {

  var subscriber = new red5prosdk.RTCSubscriber()
  subscriber.init({
    host: 'yourstreammanager.com',
    app: 'streammanager',
    streamName: 'mystream_2',
    protocol: 'wss',
    port: 8083,
    connectionParams: {
      host: '10.0.0.0',
      app: 'live'
    },
    subscriptionId: 'subscriber-' + Math.floor(Math.random() * 0x10000).toString(16)
  })
  .then(function () {
    subscriber.subscribe()
  })
  .catch(function (e) {
    console.error(e)
  })

})(window.red5prosdk)

When the Stream Manager proxy is used in the top-level configuration the app property is assigned as streammanger and a connectionParams object is provided to detail the endpoint to proxy to.

Subscribing with Flash

When using Flash the video streams cannot be dynamically switched by the Red5 Pro Server. The Adaptive Bitrate control switching logic is implemented within the Flash-based subscriber of the Red5 Pro HTML SDK. The logic to downgrade the stream is based on the InsufficientBandwidth event which is sent by the server when the client is recognized to not have enough bandwidth for the current stream variant. On the other hand, the logic to upgrade variants is a time-based polling solution in the lapse of having received InsufficientBandwidth events. This logic is triggered by providing the provision configuration, that was provided to the Stream Manager, and an optional stream switch setting configuration to the Flash-based subscriber.

The existing provision can be retrieved from the Stream Manager with the following call:

Method:GET

URL: https://yourstreammanager.com/streammanager/api/3.1/admin/event/meta/live/mystream?accessToken=myaccessToken

Where:

  • yourstreammanager.com: is the host of your Stream Manager.
  • mystream: is the top-level name of the stream.
  • accessToken: is the token defined in the Stream Manager configuration.

A successful response will return a JSON object similar to the following:

{
  "name": "mystream",
  "scope":"live",
  "data": {
    "meta": {
      "authentication": {
        "password": "",
        "username": ""
      },
      "qos": 3,
      "georules": {
        "regions": ["US", "UK"],
        "restricted": false,
      },
      "stream": [
        {
          "level": 3,
          "name": "mystream_3",
          "properties": [
            "videoBR": 128000,
            "videoHeight": 180,
            "videoWidth": 320
          ]
        },
        {
          "level": 2,
          "name": "mystream_2",
          "properties": [
            "videoBR": 512000,
            "videoHeight": 360,
            "videoWidth": 640
          ]
        },
        {
          "level": 3,
          "name": "mystream_1",
          "properties": [
            "videoBR": 1000000,
            "videoHeight": 720,
            "videoWidth": 1280
          ]
        }
      ]
    }
  }
}

In addition to the provision, it is also necessary to request the edge endpoint information from the Stream Manager and specify the stream variant that you want to subscribe to. The request is as follows:

Method:GET

URL: https://yourstreammanager.com/streammanager/api/3.1/event/live/mystream_2?action=subscribe

Where:

  • yourstreammanager.com: is the host of your Stream Manager.
  • mystream_2: is the name of the stream variant that you want to subscribe to.

A successful response will look like the following:

{
  "serverAddress": "10.0.0.0",
  "scope": "live",
  "name": "mystream_2"
}

Once both requests have been made successfully it is possible to initialize the RTMPSubscriber. Assuming that the JSON object returned by the provision request has been assigned to a variable named provision , then the RTMPSubscriber can be initialized as follows:

(function (red5prosdk) {

  var subscriber = new red5prosdk.RTMPSubscriber();

  // On embed success, provide the provisioning and target level.
  subscriber.on('FlashPlayer.Embed.Success', function () {
    subscriber.setABRVariants(provision.data, 2);
  });

  subscriber.init({
    host: '10.0.0.0',
    app: 'live',
    streamName: 'mystream_2',
    protocol: 'rtmp',
    port: 1935,
    useAdaptiveBitrateController: true
  })
  .then(function () {
    subscriber.subscribe()
  })
  .catch(function (e) {
    console.error(e)
  })

})(window.red5prosdk)

It is important to note that useAdaptiveBitrateController needs to be specified to notify the Flash client that it will require the logic for Adaptive Bitrate control.

ABR API for Flash-based subscriber

An API is provided by the Flash-based subscriber to allow to start, stop and switch stream variants explicitly. The following API calls are provided:

  • setABRVariants
  • setABRLevel
  • setABRVariantUpgradeSettings
  • startABRController
  • stopABRController

setABRVariants

Requests to use the provided ABR variants when performing Adaptive Bitrate Control.

Arguments:

  • abrVariants: Object - Provisioning Variants object
  • level: int - The target level to use for playback

setABRLevel

Requests to set a specific ABR level thus overriding the value chosen by the SDK.

Arguments:

  • level: int - The level to set the ABR based on the variants provided in setABRVariants
  • firm: Boolean: Flag to disable the automatic level switching performed by the Adaptive Bitrate Controller.

setABRVariantUpgradeSettings

Provides the stream upgrade settings that the Adaptive Bitrate Controller should use when upgrading a stream that was previously downgraded.

Arguments:

  • abrVariantUpgrades: Object - A configuration object that defines the associated retry limits based on the levels of the provision variants.

An example of abrVariantUpgrades object is:

{
  minimumDowngradePlaybackSpan: 2000,
  upgrade: [
    {
      level: 1,
      retryTimeout: 0
    },
    {
      level: 2,
      retryTimeout: 2000
    },
    {
      level: 3,
      retryTimeout: 4000
    }
  ]
}

startABRController

Request to start the Adaptive Bitrate Controller from automatically downgrading and upgrading the streams based on the status of the network and bandwidth available.

stopABRController

Request to stop the Adaptive Bitrate Controller from automatically downgrading and upgrading the streams based on the status of the network and bandwidth available.

Subscribing with HLS

Adaptive Bitrate is supported also by HLS subscribers. In this case it is necessary to subscribe to the m3u8 stream with the top-level stream name. The stream variants are defined in the m3u8 manifest and the HLS playback element has the capability to switch between them based on the conditions of the network.

As for WebRTC and Flash the first step involves requesting an edge endpoint from the stream manager using for the stream name one of the existing variants. The call is as follows:

Method:GET

URL: https://yourstreammanager.com/streammanager/api/3.1/event/live/mystream_2?action=subscribe

Where:

  • yourstreammanager.com: is the host of your Stream Manager.
  • mystream_2: is the name of the stream variant that you want to subscribe to initially.

A successful response will look like the following:

{
  "serverAddress": "10.0.0.0",
  "scope": "live",
  "name": "mystream_2"
}

This data is then used in the initialization configuration for the HLSSubscriber which will look like the following:

(function (red5prosdk) {

  var subscriber = new red5prosdk.HLSSubscriber()
  subscriber.init({
    host: '10.0.0.0',
    app: 'live',
    streamName: 'mystream',
    protocol: 'html',
    port: 5080
  })
  .then(function () {
    subscriber.subscribe()
  })
  .catch(function (e) {
    console.error(e)
  })

})(window.red5prosdk)

This will make the Red5 Pro SDL request to subscribe to the HLS stream at:

http://10.0.0.0:5080/live/mystream.m3u8

Stream Manager Rest API Documentation

See Rest API for Stream Provisioning for more details on the above calls.

Transcoder Examples

Publish Stream Manager Proxy - Transcoder POST and Publish Request

Subscribe Stream Manager Proxy Transcoder, RTC

Subscribe Stream Manager Proxy Transcoder, RTMP

Subscribe Stream Manager Proxy Transcoder, HLS