September 7, 2015

Processing RTMP, RTSP, SLDP and Icecast pull streams per request

Live streaming of video over Internet gives a high load on the communication channels. To reduce channels load, and to increase robustness of live streaming the load balancing techniques are used (read the "HLS, DASH and Icecast streaming load balancing" article for details).

Configure pull RTMP only for requested streams

During origin and edge servers configuration for RTMP streams you can use 2 types of restreaming:
  1. Publish RTMP stream from an origin (or encoder) to an edge server.
  2. Edge server can pull RTMP streams. You need to specify IP or hostname of origin server (or encoder).
For RTSP protocol the settings are performed using the same techniques.
For SLDP, Icecast and SHOUTcast you can use only pull.

You can read more detailed information about the creating of your live streaming network in the "Building RTMP live streaming network via Nimble Streamer" article. In current article we are going to show how to decrease network load by setting up RTMP, RTSP, SLDP and Icecast streams pull by request via Nimble Streamer.


Bandwidth optimization


In large networks not all your streams are requested by all users all the time. But all of these streams continue to be pulled from origin server, that is why fee for the traffic is continuously charged. 
Nimble Streamer allows to optimize usage of bandwidth via setting up RTMP, RTSP or Icecast streams to be pulled only if users request to view these channels via one of supported output protocols:

  • HLS;
  • MPEG-DASH;
  • RTMP;
  • RTSP;
  • SLDP;
  • Icecast;
  • MPEG-TS.

For example, large content providers can offer thousands of media URLs of live video, but only part of them will be requested by users at certain periods of time. So if 1 viewer is requesting an RTSP stream via HLS and another one requests it via Icecast, then 1 stream will be pulled via RTSP and transmuxed into HLS or any other protocol for output as long as these viewers watch the stream.

The same approach is relevant when using CCTV cameras which transmit video streams over RTSP protocol. Usually in this case only about 20-30 cameras are displayed on the monitor, but overall cameras count may be several hundreds (i.e. for the systems like "Safe town").

Setup procedure


You can perform the advanced Live pull settings setup via Nimble Streamer. This setup allows you to pull only those streams which are requested by viewers and also you can configure media server to pull any streams for the certain application.

We'll show RTMP as example but you can apply this to RTSP and Icecast/SHOUTcast as well.

To perform these settings go to "Nimble Streamer" -> "Live streams settings" -> "Live pull settings" and click the "Add RTMP URL" button.



Specify the pulled  live video URL in the appeared dialog. You can specify fallback URL (optional). Specify the "Output application" and "Output stream" fields.

For per-request pull of streams, you also need to specify the "Idle time" value. If the viewers have stopped watching the stream then Nimble Streamer will keep pulling the stream from the origin for this period of time and if nobody else connects to it, the transmission from the origin is stopped. If you need constant stable connection, you don't need to use this option.



You can setup receiving of any RTMP, RTSP or Icecast streams by checking the "Dynamic stream name" and specifying only application name in the Primary URL (i.e. rtmp://remotehost:1935/live). In this case, any stream belonging to the specified application (e.g., live) is pulled from a source and is processed as described above. This allows you to specify the entire set of possible streams instead of one stream, which greatly simplifies the configuration.

Once the server is configured, the first user requesting the video over HLS, RTMP or other supported protocol will initiate a pull session.

This configuration can significantly save the bandwidth if your streams are rarely used. At the same time latency, associated with the resumption of the stream, will not exceed 200 ms. If your streams are constantly demanded, the enabling of this function will not give the advantages. For more details about live streaming latency please read "Achieving low latency for live streaming using Nimble Streamer" blog article.

If you want to speedup the playback start time, you may also consider improving buffer setting. This origin-side setup will allow improving this behavior.

Security settings


However, you should be very careful when using dynamic stream names. The application could have several hundreds of streams (e.g. video cameras), and not all of these streams need to be pulled from the source.
This creates the risk that users can force media server to pull a huge amount of streams by their actions. It can cause of unnecessary using of bandwidth  and computational resources. In order to prevent the use of such scenario, you can create the WMSAuth rule, which allows to take and to play only those streams that the user is allowed to access. For more information about restricting access by streams names please read "Hotlink protection with stream-based signature" article.


Adaptive bitrate


Having the input streams, you can transform their content using various techniques using our Live Transcoder for Nimble Streamer to transform. It has high performance and low resource usage.

DVR - recording and playback


Both RTMP and RTSP may be recorded for further playback via HLS and MPEG-DASH using the DVR feature set. Read this article for setup details.


Related documentation


13 comments:

  1. This is amazing. Question -- am I able to add and remove RTSP URLs via your API? The closest I could find is https://wmspanel.com/api_info#toggler=rtsp_live_pull3 but this does not appear to have the same settings (such as Idle Time). I have hundreds of possible RTSP sources at any given moment. When one is needed, I'd like to add it via API into Nimble, stream, and then once disconnected, remove it from Nimble. Is that possible?

    ReplyDelete
    Replies
    1. Thank you for indicating this issue, we'll add pull per request to respective RTSP API methods.

      Delete
    2. Wow, thanks for the fast response - I look forward to the update

      Delete
    3. No problem. Feel free to try this feature in action to see what else you might need.

      Delete
    4. OK, it's been added, check out the API description. Thanks for reporting!

      Delete
  2. Нет поля для описания url, что очень неудобно при большом количестве.

    ReplyDelete
    Replies
    1. О каком именно описании вы говорите? Там есть Multiple edit - оно позволяет задавать любое число потоков в текстовом виде.

      Delete
  3. so when you use pull from the edge server does it initiate a pull for every user that tries to watch the stream, or does it only pull on the first user that tries to watch? so you only have on connection going between the origin and the edge?

    ReplyDelete
    Replies
    1. The stream is pulled only once from each edge server. The rest of the processing and clients' handling is performed on edge server side.

      Delete
  4. So when you use pull does it only have one connection between the origin and the edge server when you have multiple users viewing the same stream? so multiple users can watch the same stream without killing the connection between the origin a the edge server?

    ReplyDelete
    Replies
    1. Justin,

      That's correct - the stream is pulled to the edge just once and then transmuxed for viewing by multiple users.
      If there are no viewers' connections for some time - bigger than defined idle time - the connection to origin is lost. Once new viewer requests it from edge, it will be established again.

      Delete
  5. How long until failover to backup stream? Is this the same as idle time?
    I have tested it, switched off primary stream at source, flowplayer took 60 seconds for stream to continue playback.
    Any way for this to be quicker, without changing timeout to very low? E.g., 15 seconds?

    ReplyDelete
    Replies
    1. Can you send an example of such failing stream to https://wmspanel.com/help ?

      Delete