The State of Streaming Protocols - December 2014

WMSPanel team continues analyzing the state of streaming protocols. December showed a lot of views which means more of interesting content to look at.

We expectantly see the increase of HLS share (it's 70% now). The share of RTMP (22%) has been decreasing along with RTSP (2.5%) while progressive download showed good growth (~5%).

All these metrics calculations are based on 3 billion views.

The State of Streaming Protocols December, 2014

You can compare that with November stats below.

Cross-domain policy and access control in Nimble Streamer

Protecting web clients against vulnerabilities is one of the core features for safe web browsing. This is why popular platforms use various mechanisms to improve the security. This also includes video playback scenarios. Typically it's covered cross-origin resource sharing (CORS) mechanisms. Nimble Streamer needs to handle those cases.

Cross-domain policy file

For instance, many web media players use the "crossdomain.xml" file. It's a cross-domain policy file which gives the player permission to talk to servers other than the one it's hosted on.

As per Adobe, a cross-domain policy file is an XML document that grants a web client permission to handle data across multiple domains. When a client hosts content from a particular source domain and that content makes requests directed towards a domain other than its own, the remote domain would need to host a cross-domain policy file that grants access to the source domain, allowing the client to continue with the transaction.

Nimble Streamer allows specifying this file content. It is done via configuration file located at /etc/nimble/nimble.conf . Please refer to Nimble config file format for more details.

Use the following parameter to specify the domain within the XML file:

crossdomain_xml_allow_access_from_domain = <your.domain>

E.g. crossdomain_xml_allow_access_from_domain = wmspanel.com

If you'd like to define complete file content, you can create crossdomain.xml with XML based on Adobe's spec examples, like this:

<?xml version="1.0"?><cross-domain-policy><allow-access-from domain="*" secure="false"/></cross-domain-policy>

Once it's ready, save it to local directory and point Nimble to it via crossdomain_xml parameter:

crossdomain_xml = /etc/nimble/crossdomain.xml

The file content will be returned to a client as soon as it's requested.

Cross-origin resource sharing

Another technique is cross-origin resource sharing. Players may require Access-Control-Allow* headers in server responses. You can use the following parameters for that:

access_control_allow_origin = <some value>
access_control_allow_credentials = <some value>
access_control_expose_headers = <some value>
access_control_allow_headers = <some value>

Some of possible values are

access_control_allow_origin = *
access_control_allow_credentials = true
access_control_expose_headers = Content-Length
access_control_allow_headers = Range

or

access_control_allow_origin = http://your.domain.com

You may refer to W3C to get more details.

Important: to apply config changes, please re-start Nimble instance by running:

sudo service nimble restart

Please read more about config file format here.

P2P streaming

You can also see example of CORS headers usage in the StreamRoot P2P streaming article featuring Nimble Streamer.

Related documentation

Nimble Streamer, Nimble Streamer API, Nimble Streamer configuration, Nimble Streamer performance tuning, SSL support for HLS, DASH, Icecast and MPEG-TS streaming,

Geo-location statistics for states and regions

WMSPanel reporting for media servers of various types has several types of metrics. Geo-location reporting is one of the most popular ones. It's part of daily statistics for particular data slice and it shows total number of connections established within selected dates range.

Today we introduce a highly anticipated improvement - geo-location stats how have extra information about regions within countries. Best example is a state within countries the United States of America or Brazil.

You can go to Reporting -> Geo stats menu to see a reporting page. This page contains a map representing views share and a list of countries, each having total number of connections and percents of share. You may select dates range to report and also export your data via CSV file.

List of countries in geo-location reporting.

Clicking on country name shows list of regions (e.g. states) list, each having total connections number and percentage.

List of states for USA in geo-location reporting.

Clicking on a region shows the list of cities in it, each having its own stats.

List of cities of a country in geo-location report.

With the additional regions metrics you will have full picture of your audience in any given country.

Read more about WMSPanel streaming reporting. Contact us if you have any questions about it.

Related documentation

End user reporting for Wowza, Daily statistics, ISP networks report, Data slices for statistics, Devices and Players report for Wowza, Streamed slices for Wowza, Screencast for daily statistics, Statistics import API, Geo and IP range restriction for Wowza, Nimble Streamer geo-location restriction

This product includes GeoLite data created by MaxMind(c), available from http://www.maxmind.com

SSL support for HLS, WebRTC, Icecast, DASH, MPEG-TS, SLDP

Secure streaming is required in several scenarios in our customers' environments. This is why we are working on implementing security feature set. One of the high-demand features is SSL streaming for HLS, MPEG-DASH, MPEG-TS, Icecast, progressive download and SLDP via Nimble Streamer. In this case streams are available via HTTPS protocols stack.

Nimble Streamer team has implemented this feature.

To set up HTTPS streaming, you need to generate SSL certificate first. Usually SSL certificates are purchased by some provider like GoDaddy and these companies provide plenty of information about this process.

In this article we assume:

• you already have a certificate for further setup,
• your certificate and its key are located at your server and 
• they are ready for further usage.

You will need to make changes to Nimble Streamer settings to make it work for your media streaming. These settings are stored in /etc/nimble/nimble.conf file, you should read configuration parameters reference page for more details about parameters' control, just click on SSL tag.

Follow the steps described below to enable SSL-powered streaming.

Notice that Nimble Streamer supports Lets Encrypt certificates. Read this article for full setup details.

1. General parameters

ssl_port parameter - this is port number for SSL connections. Example:

ssl_port = 443

By default, Nimble Streamer handles connections via the port specified in config at "port" parameter. Usually it's port 8081. If you need Nimble Streamer to handle connections via SSL only, please set this parameter to 0, like this:

port = 0

If it has some other value, Nimble still handles streaming connections through 2 ports via both HTTP and HTTPS. If you remove "port" parameter, Nimble will use default value "8081"

ssl_protocols parameter specifies what SSL protocols are used. It requires the list of protocols separated by spaces, e.g.

ssl_protocols = TLSv1 TLSv1.1 TLSv1.2

Full list is: SSLv2, SSLv3, TLSv1, TLSv1.1, TLSv1.2, default protocols are TLSv1, TLSv1.1 and TLSv1.2.

HTTP/2 is supported by Nimble Streamer and it can be used only when streaming over HTTPS. In order to process HTTP/2 requests, use this parameter:

ssl_http2_enabled = true

Read this article for more details on HTTP/2 usage.

2. Certificates and domains

2.1 Single certificate

The following parameters define the usage of SSL certificate for your Nimble Streamer instance:

• ssl_certificate is full path to certificate located at your server;
• ssl_certificate_key is full path to certificate's private key located at your server;
• ssl_certificate_key_pass - if you use encryption for your certificate key, you need to specify a password here. This is optional parameter, so if you don't use encryption, just don't add it into the config.

Here's an example of SSL config parameters:

ssl_port = 443
ssl_certificate = /conf/cert/default.crt
ssl_certificate_key = /conf/cert/default.key.dec

These settings will work for entire server.

2.2 Multiple certificates

Besides server-wide certificate, you can assign specific certificates and keys to a certain set of domains. This is handled by adding appropriate ssl_server sections to the config. Each section may have the following items.

• server_name describes server name patterns to apply certificate to - see the description below;
• ssl_certificate, ssl_certificate_key, ssl_certificate_key_pass and ssl_http2_enabled as described above.

server_name element defines the host names where the certificate will be applied to. Multiple names are separated by spaces.

There are 3 types of name patterns you may use:

• exact match works when you specify the exact name of the host you want it to work for.
• wildcard match name may contain an asterisk only on the name’s start, and only on a dot border. It will match all domains which start with anything before the asterisk with the exact name after it. So wildcard "*." may be used only once in one name pattern. Example: *.wmspanel.com matches for all <anything>.wmspanel.com host names.
• regexp match starts with '~' character and consists of a Perl-compatible regular expression describing the name pattern for all required host names.

You may have any number of ssl_server sections in your Nimble Streamer config. The first section will be the default one in case of multiple server_name parameters match the same host.

Here's an example of SSL config parameters:

ssl_port = 443
ssl_certificate = /conf/cert/default.crt
ssl_certificate_key = /conf/cert/default.key.dec
ssl_server {
  server_name = *.wmspanel.com ~^.*\.wmspanel.com$
  ssl_certificate = /conf/cert/wildcard_wmspanel_com.crt
  ssl_certificate_key = /conf/cert/wildcard_wmspanel_com.key.dec
}
ssl_server {
  server_name = wmspanel.com
  ssl_certificate = /conf/cert/wmspanel_com.crt
  ssl_certificate_key = /conf/cert/wmspanel_com.key.dec
  ssl_http2_enabled = true
}

Notice that ssl_http2_enabled needs to be specified in each individual ssl_server section if you need to use it.

3. Apply config

To apply config changes please re-start Nimble by running:

sudo service nimble restart

You can read more about Nimble Streamer parameters and their usage in this reference article.

4. SSL cache and control API 

Nimble Streamer has certificates cache. So it takes each SSL certificate from cache if it's already there, and it doesn't re-load it on each connection.

If you need to reload certificates in real-time without re-starting Nimble Streamer, you may use Nimble API. It has /manage/reload_ssl_certificates method for this. This method receives POST request and Nimble Streamer re-loads all current certificates. If they are loaded successfully, all new connections will use this new set, even though some of them may have been used previously. Currently connected clients will keep previous certificates for their current sessions until they are closed.

Example of API call:

curl -X POST http://127.0.0.1:8083/manage/reload_ssl_certificates

Let's Encrypt Certbot

Nimble Streamer supports Lets Encrypt certificate and allows Certbot auto-renewal. Read this article for full setup details.

Troubleshooting

Some browsers or client software may fail to recognize your certificate even though it's valid. In this case you may need to get SSL certificate chain (root and intermediate certificates) combined into a single .crt file and use that file with Nimble Streamer. If you use Linux, you can do that by concatenating both files using this command:

cat your_site_certificate.crt root_certificate.crt > your_site_chained_certificate.crt

Please feel free to install Nimble Streamer to try this and other security-related features in action. Contact us in case of any questions or issues.

Follow us in social media to get updates about our new features and products: YouTube, Twitter, Facebook, LinkedIn, Reddit, Telegram

Related documentation

Nimble Streamer, Hotlink protection for Nimble Streamer, Paywall for Nimble Streamer, Live streaming, VOD streaming, SLDP low latency streaming,

MPEG-DASH hotlink protection and paywall

Nimble Streamer team keeps working on MPEG-DASH feature set. In addition to MPEG-DASH VOD streaming and RTMP to DASH live transmuxing, we've continued implementing features already existing for HLS.

One of the most popular feature sets of Nimble Streamer and WMSPanel is content protection. It allows creating easy-to-use paywalls and embedding them into existing websites. Major available features are hotlinking protection and pay-per-view framework.

Hotlinking protection

Hotlinking protection allows preventing "links hi-jacking". When you publish some media behind the paywall, anyone can take the direct URL to content and pass it to anyone else. Nimble Streamer allows preventing that. This is how it works.

1. You modify a webpage which shows a media link to include a "signature" - the URL parameter which is based on a secret password and viewer's IP address
2. You also set up protection rules in WMSPanel where you describe which streams are affected and what password will be used.
3. When a viewer requests the media from the media server, Nimble Streamer checks the signature against the viewer's IP and encrypted password. If they match, the connection is established, if they don't then it will be forbidden.

This scenario works already for HLS and progressive download already. Now it works perfectly for MPEG-DASH.

Hotlinking protection for MPEG-DASH workflow diagram.

Making MPEG-DASH from RTMP: ABR streaming via Nimble Streamer

Nimble Streamer team keeps exploring MPEG-DASH capabilities. In addition to MPEG-DASH VOD streaming introduced, we've continued moving towards live streaming support. Today we release RTMP to MPEG-DASH transmuxing. Nimble Streamer may take published RTMP as well as pull stream from RTMP source and transmux it into DASH in live mode.

To use this capability you need to install Nimble Streamer and then define RTMP settings - either specify RTMP publish settings or RTMP pulled streams. Once it's done, you'll be able to get live MPEG-DASH streams for single bitrates as well as make ABR streams from them.

NEW: Nimble Streamer supports AVC/H.264, HEVC/H.265 and AV1 codecs for re-packaging into MPEG-DASH. Read more about Enhanced RTMP spec adoption for more information.

Let's go step by step and see how it works.

1. Install Nimble Streamer

Use this installation instruction.

To take RTMP streams for transmuxing, you need to specify available sources. After that Nimble will pick them up and start producing HLS chunks for immediate use.

Go to Nimble Streamer -> Live Streams Set Up menu to see list of available servers.

Available Nimble Streamer instances.

Then choose a server to make settings. Most of actions may be applied to multiple servers at once so you can choose any of them.

Incoming streams settings.

Now click on RTMP settings button to proceed with any of 2 possible scenarios:

a) get published RTMP streams;

b) pull RTMP streams.

You may combine both scenarios and process both types of incoming streams to get both HLS and MPEG-DASH outgoing streams.

2.a Publish scenario: Set up RTMP publish settings

Applications' settings

Going into RTMP settings page, first you'll see several tabs. First one you need is Global. These are global server settings. These are as follows.

• Default chunk duration used for outgoing streams
• Protocols which will be produced - you can generate all 4 supported types or just one of them, it does not affect the performance much.
• Push login and password for published streams - they will be used by default for published streams.

Global server settings.

You may also define individual applications' settings. Go to Applications tab to add new apps. Each app has the same set of fields as Global server settings.

Individual applications' settings.

New application settings.

You may apply new application settings to several Nimble servers instances. Just click on their names' checkboxes in the dialog - the setting will be applied to each server within a few seconds.

Add interface

To make Nimble Streamer capable of getting published RTMP streams, it needs to listen to a specific interface - address and port. Go to Interfaces tab and click on Add interface.

Interfaces list.

You'll see a dialog for specifying an address and a port to listen to. You may leave IP address blank, in this case Nimble will listen to all IP addresses available.

You may also apply new settings to multiple servers to convenience of administration.

Adding new interface to process RTMP publishing.

If you have only published streams, then you can move to step 3.

2.b Pull scenario: Set up RTMP pull settings

If your streaming content is available via available RTMP streams, you may pull them into Nimble instances for further transmuxing. To make proper settings, go to Live pull settings tab. 

Pulled RTMP streams list.

There you click on Add URL button to see new dialog for adding new stream to transmux. There you enter:

• URL - the address of RTMP stream.
• Fallback URLs - if you have multiple sources of the same stream, you may specify them to make robust streaming, so if main stream goes down, secondary streams could be used.
• Application and stream are the the names which will be used for DASH stream URL.

As already mentioned in other settings, you may apply this setting to multiple servers - just click on their names' checkboxes.

New pulled RTMP stream settings.

Once it's saved, you'll see it in settings list.

New stream in the list.

3. Outgoing streams

Having incoming streams defined and processed, you may now use the results of Nimble Streamer transmuxing for streaming your content via MPEG-DASH and any other protocol which you selected in global or application settings, e.g. HLS.

Click on Outgoing stream area on a chart or Outgoing link on top of the setup area. You will see all streams that are currently processed and ready for usage. Each stream has

• status, 
• names of servers which have this stream running, 
• stream name for playback URL,
• video and audio parameters and options,
• link for getting playback URL - it's a question mark.

Available outgoing streams list.

To use the outgoing stream for playback, click on question mark icon to see Sample URL for player dialog.

MPEG-DASH and HLS stream names.

Here you see links for 2 protocols which you defined for this server in global settings - MPEG-DASH and HLS.

With streaming URLs ready for further use, you may now add them to your player. MPEG-DASH streaming of Nimble Streamer was tested with DASH reference player as well as with Bitmovin's bitdash™ player. If you find any other players more suitable for you - please share your experience with us.

4. Adaptive bitrate for MPEG-DASH streams

Another feature set common across HLS and MPEG-DASH streaming is adaptive bitrate streaming. It allows you to give a stream containing sub-streams with different bitrates so a player or a viewer could select appropriate bitrate according to their network conditions.

To define ABR streams, click on Adaptive stream area on a chart or ABR link on top.

ABR settings page.

Click on Add ABR stream from RTMP sources button.

Here you need to define the following fields:

• ABR application name and stream name - they will be used for playback by your viewers.
• Source application name and stream name - you can add several sources from currently available single-bitrate streams. Just start typing names to see gestures listing available sources.
• Streams sorting in playlist:
• Manual - as defined in list;
• By bandwidth - descending;
• By bandwidth - ascending.

As in other Nimble settings, you can select multiple servers to apply these settings to. Thus you can specify ABR stream for your entire infrastructure.

Having set up the stream, you will see it in the list. Once it's synced up, it'll be shown in "online" state. To start using it, click on question mark icon to see Sample URL for player dialog.

URLs for MPEG-DASH and HLS ABR streams.

Use these URLs for further playback in your DASH-capable or HLS-capable players as previously described on step 3.

You may also consider using RTMP streaming API to control this behavior remotely.

What's next

If you need to change content before transmuxing into ABR, use our Live Transcoder for Nimble Streamer to transform. It has high performance and low resource usage.

Using Nimble Streamer in your streaming infrastructure, you can get DASH streaming statistics via WMSPanel reporting SaaS. It has built-in support so you don't need to parse logs or anything like that - Nimble will send required metrics to central service for your convenience.

Also take a look at video tutorial about RTMP setup.

It shows basic scenarios with RTMP processing.

WMSPanel is currently an associate member of DASH Industry Forum. We will extend DASH feature set with all scenarios available for HLS. This includes features like live streaming from MPEG-TS (both UDP and HTTP). Please contact us if you need to cover some specific use cases or if you just have any feedback about DASH handling in Nimble Streamer.

Related documentation

Live Streaming features in Nimble, Live Transcoder for Nimble Streamer, RTMP feature set,  DASH industry forum, MPEG-DASH VOD streaming, Hotlink protection and paywall for MPEG-DASH, WMSPanel API reference, Build streaming infrastructure with Nimble Streamer, 

MPEG-DASH support for VOD playback in Nimble Streamer

Through past several years, MPEG-DASH has gone a long path from a basic concept to industry standard. Thanks to contributions from a lot of companies and stakeholders, this technology can now be used in a variety of scenarios, including video on demand.

Nimble Streamer team is up for HTTP streaming technologies like HLS or Icecast. So moving towards DASH is a great step forward to better user experience for our customers who would like to try advantages of this HTTP-based protocol.

Today we introduce VOD playback for MPEG-DASH in Nimble Streamer. Is allows transmuxing MP4 files to DASH stream on-the-fly with very low resources consumption. It supports both time-based and number-based manifests for VOD DASH.


You can follow these easy steps to set up DASH VOD streaming.

1. Install Nimble Streamer

Use this installation instruction.

2. Prepare content

Upload your content to a designated location on the server. This needs to be MP4 file with H.264, H.265 or AV1 video and AAC or MP3 audio.

The user called "nimble" must have read access to this directory and its contents. Let's say it would be /home/user/video directory

3. Set up VOD streaming

Click on Nimble Streamer -> VOD & re-streaming routes menu to open streaming routes setup page.

Click on Set up VOD streaming button to see the following dialog.

Adding VOD transmuxing route for DASH, HLS and progressive download.

In section 1 "Where incoming requests are coming" you may see the following fields.

• Path field contains the URL name part which will then be used for accessing streams. You need to enter some value here, it shouldn't be blank because it's used for stats calculation purposes. It's /vod/ in this example. 
• You may also set up Domain if you'd like this route to process only specific domains. You may leave it blank if you don't plan putting any limitations.

In section 2 you need to fill in Path field with full path to your content in your file system.

Then you need to select which Nimble Streamer instances will get these settings. So you may apply them to any number of instances at once in just a few clicks.

Once the route is set up, you may request any of the uploaded files via any protocol you need:

• http://server_IP:8081/vod/sample_file.mp4/manifest.mpd - MPEG-DASH time-based manifest
• http://server_IP:8081/vod/sample_file.mp4/manifest_number.mpd - MPEG-DASH number-based manifest
• http://server_IP:8081/vod/sample_file.mp4/playlist.m3u8 - HLS
• http://server_IP:8081/vod/sample_file.mp4/playlist_fmp4.m3u8 - fMP4 HLS
• http://server_IP:8081/vod/sample_file.mp4 - progressive download

You may also change default port 8081 to any other port by changing server config.

If you'd like to have better viewing experience you may also consider using adaptive bitrate for VOD MPEG-DASH by using SMIL files.

Take a look at this article to see how you can use WebVTT, SRT and TTML subtitles with your VOD HLS streams

4. Stream the content

With streaming URLs ready for further use, you may now add them to your player. MPEG-DASH streaming of Nimble Streamer was tested with DASH reference player as well as with Bitmovin's bitdash™ player. If you find any other players more suitable for you - please share your experience with us.

Using Nimble Streamer in your streaming infrastructure, you can get DASH streaming statistics via WMSPanel reporting SaaS. It has built-in support so you don't need to parse logs or anything like that - Nimble will send required metrics to central service for your convenience. Check also MPEG-DASH live streaming via RTMP transmuxing.

Troubleshooting

Nimble Streamer internal transmuxer buffer sometimes may not be enough to fit generated HLS chunk. This may lead to this error in Nimble Streamer logs:

[tmux1] E: buffer is too small to generate

By default the buffer is 40MB. You can increase it using "vod_transmuxing_buffer". Read Nimble Streamer configuration description for config setup details.


You can also take a look at the video where Jan Ozer of Streaming Learning Center and Yury Udovichenko of Softvelum discuss pros and cons of dynamic packetizing of live, VOD and DVR content. It gives more details about Nimble Streamer VOD streaming.

Related documentation

Nimble Streamer, MPEG-DASH live streaming from RTMP, DASH industry forum, HLS VOD streaming, HLS live streaming, Hotlink protection and paywall for MPEG-DASH

The State of Streaming Protocols - November 2014

WMSPanel team continues analyzing the state of streaming protocols. November showed a lot of views which means more of interesting content to look at.

We expectantly see the increase of HLS share (it's 69% now). The share of RTMP (23%) remains the same while RTSP (4.5%) has decreased. MPEG-DASH is showing a growth still having less than 1% overall.

All these metrics calculations are based on 1.88 billion views.

The State of Streaming Protocols November, 2014

Re-streaming SSL-protected HLS

Re-streaming of HLS streams from origin via edge server is one of the most popular features of Nimble Streamer. You can use any origin of HLS media and make it available to large number of viewers because Nimble Streamer is very light-weight and fast. So regardless of origin performance, your infrastructure will be able to handle thousands of viewers with minimum investments.

You may use secure origins so they have SSL protection for HLS traffic and use HTTPS for their connections. Now Nimble Streamer is able to use SSL to communicate to these origins for HLS re-streaming.

The output would be the same as Nimble has now. To make it use SSL, go to Re-streaming setup dialog and click on Use SSL checkbox. for the routes where origin uses HTTPS as shown below.

Set up HLS SSL re-streaming for Nimble Streamer.

If you'd like to stream the received content via SSL for your viewers, Nimble can do that as well. Read this article for details.

Re-streaming is just one of the use cases where Nimble Streamer is used. You can set it up as HLS origin for multi-bitrate live streaming as well as an origin for VOD streaming. It also has RTMP re-publishing feature set which allows setting up efficient live re-streaming across multiple origin-edge servers.

If you need more sophisticated protection for HLS, please consider using HLS AES-128 DRM encryption supported by Nimble.

Contact us if you have any questions regarding this or other Nimble Streamer features.

Related documentation

Nimble Streamer, HLS re-streaming via Nimble, Streaming VOD with Nimble Streamer, Pull RTMP to HLS transmuxing, RTMP republishing, Pay-per-view for Nimble Streamer,

Icecast and MPEG-TS hotlinking protection

WMSPanel paywall framework is the foundation for a variety of content protection systems within environment of our customers. It's available for both Nimble Streamer and Wowza Streaming Engine.

As Nimble Streamer has Icecast and MPEG2-TS transmuxing capabilities we also made changes to the paywall framework to support that. How you may specify Icecast and MPEG-2 TS in the Links re-publishing protection section. This is primarily used for streams hotlink protection as well as for pay-per-view processing.

Protecting HLS, progressive download, Icecast and MPEG-TS against links re-publishing.

Nimble Streamer also provides unique capabilities like domain lock and stream-name based protection for your content.

Please feel free to install Nimble Streamer to try this and other security-related features in action. If you need any help, check our FAQ first and contact us in case of other questions.

Related documentation

WMSPanel, Nimble Streamer, Hotlink protection for Nimble Streamer, WMSAuth security feature set articles, Paywall for Nimble Streamer, Paywall FAQ, MPEG2-TS transmuxing via Nimble Streamer

Nimble Streamer cache performance

Nimble Streamer is created with high performance in mind. Being a native Linux application, Nimble is fast and has low resources consumption. This is why it is chosen for robust high-availability infrastructures and high-speed transmission use cases.

So our customers want to learn more about hardware configuration to be best fit for usage with Nimble Streamer. They also want to know what is their existing hardware capable of when using Nimble.

The description below covers these and other aspects of Nimble Streamer performance tuning in terms of caching. We will refer to Nimble config and its parameters - it is described in Nimble Streamer configuration page.

Calculating RAM amount for live streaming cache

The most used parameter which influences the streaming process is the amount of RAM available for caching. This amount is used for HLS chunks storage.

For each stream, Nimble stores 4 chunks in cache. Once the chunk is out of the playlist, it gets timeout of 45 seconds. So additionally the cache stores several chunks and the number depends on chunk duration. If it's 6 seconds, this would be 4 + 45/6 = 4 + 7 = 11 chunks. For 10 seconds chunks this would be 4 + 45/10 = 4 + 4 = 8 chunks.
The consumed memory amount for those chunks will depend on the bitrate:

RAM size (bytes) = number of chunks * chunk duration * bitrate / 8

I.e. for 1Mbps stream with 6 seconds chunks this would be 8.25MB. If you have an ABR HLS streaming with 512Kbps, 1.5Mbps and 2Mbps, with 10 seconds chunks, your cache amount would be around 40MB.

Notice that this number does not depend on a number of simultaneous viewers.

In addition to cache size you also need to consider the RAM size which your OS will take for network processing. As you can see from this real-life example the OS itself may take a lot more than Nimble Streamer instance.

RAM cache size parameter is set up via web UI.

VOD streaming cache control

VOD streaming also requires cache settings. As in case of live streaming, by default it's located in RAM. In addition, when RAM cache is full, VOD cache starts residing in the file system. So if you're doing VOD, you need to use "Max disk cache size" parameter for setting the maximum size for VOD cache. This is done via web UI. You may also need vod_cache_timeout and vod_cache_min_storage_time parameters in config file for TTL of cached chunks.

Please read "Improving cache control for VOD re-streaming" article to learn how to control VOD streaming cache.

Examples

Let's see some common questions coming from our customers.

How many connections can be handled on 2 CPU 512MB RAM server having 1.2Mbps stream with 1Gbps network?

1.2Mbps would require about 10MB of RAM cache. Also, 1Gbps will not require changing worker threads parameter and and 1 extra CPU. So the only limit in this case is the network speed. For given bitrate, the channel will handle 830 connections.

Handling 10 streams, 512Kbps each for approximately 10K viewers, what hardware do I need for that?

From RAM cache perspective, this will take around 42MB RAM cache. However, 10000 viewers will produce 5.1 Gbps bandwidth (i.e. transmission speed). This would definitely need an additional CPU core (making it 2 CPU cores total) and worker_threads parameter set to "2".

The ABR stream has 10 seconds chunks for 240, 360, 480 and 512 Kbps bitrates. What is the cache size for that?
Sum of bitrates is 1.6Mbps. 8 chunks, each 10 seconds, divided by 8 will make 16MB of cache size. You can have 10 streams like that and still not get the limit of the cheapest 512MB RAM virtual machine. With average bandwidth of around 400Kbps, your 10Gbps network will allow you to have around 25000 simultaneous connections but that will depend on the popularity of bitrates.

More performance tuning

You could see how you can improve RAM cache, however there are several other techniques which allow Nimble Streamer to process large amounts of viewers and data.
Read our performance tuning guide to learn more about this.



Contact us if you have any further questions or need some advise on Nimble Streamer usage.

Related documentation

Nimble Streamer performance tuning, Live Streaming scenarios in Nimble, Build streaming infrastructure with Nimble Streamer, Nimble Streamer API, Nimble Streamer configuration, VOD cache control,

Nimble Streamer status API

Nimble Streamer is being used in a variety of streaming use cases and scenarios. Many customers use multiple instances for load balancing and robustness purposes. The majority of balancing techniques use geo-location approach or just round-robin balancing to split the incoming requests equally between edges. However this does not guarantee that edge servers will have equal load as some viewers many disconnect from some edges while other edge users will keep watching.

So system administrators who use Nimble Streamer need a way to get status of each server to make real-time decisions about which servers should be processing current incoming requests. Usually this means changing the URL of the media streams on the website so any player would pick it up for playback. This is the most reliable way to make balancing happen regardless of the players' implementations.

This is why we extended common WMSPanel API by adding a pull API to Nimble Streamer itself. A customer may now launch a request to any Nimble instance and get its status and live streaming settings.

Enable API for Nimble Streamer instance

To make Nimble Streamer responding to API requests, the API settings must be set up in /etc/nimble/nimble.conf configuration file. You may check config description to get details about other parameters.

All parameters mentioned below are excluded from config by default.

management_listen_interfaces
This parameter specifies which IP addresses will be used for accepting AI requests. If it's not set, the API requests are not accepted.
Examples:
management_listen_interfaces = *  - all available interfaces are used
management_listen_interfaces = 127.0.0.1, 192.168.0.1

management_port
This one specifies which port is used to listen to API requests. If it's missing, the 8082 port is used.
Example:
management_port = 8086

management_token
This parameters specifies the token (i.e. password) which is used for authorizing API requests. See Making authorized requests section below for details.
If this parameter is missing, there will be no authorization made and anyone will be able to get information.
Example:
management_token = anypassword

Making authorized requests

This is an optional step for the cases when you use management_token parameter for authorizing requests. To make authorized requests you need to make MD5 hash based on the specified token. Please refer to this code sample to see how you can generate this hash.

<?php
$salt= rand(0, 1000000);
$key = "anypassword"; // the token specified in management_token parameter
$str2hash = $salt . "/". $key;
$md5raw = md5($str2hash, true);
$base64hash = base64_encode($md5raw);
$requiest_url = "http://127.0.0.1:8082/manage/server_status?salt=$salt&hash=$base64hash"
echo $request_url;
?>

Get server basic status

This API method allows getting current number of connection and bandwidth (transmission speed) level.

Request URL
/manage/server_status

Response parameters
Connections - number of active connections
OutRate - current transmission speed, bits per seconds

ap - Available processors
scl - System CPU load
tpms - Total physical memory size
fpms - Free physical memory size
tsss - Total swap space size
fsss - Free swap space size

Request example

curl -vvv http://127.0.0.1:8082/manage/server_status

Response example
{"Connections": 10, "OutRate": 5120000, "SysInfo": {"ap":2,"scl":0,"tpms":2098434048,"fpms":775127040,"tsss":2145382400,"fsss":1707151360}}

Get RTMP connections status

Request URL
/manage/live_streams_status

Note: /manage/rtmp_status is considered deprecated.

Response parameters

• app - name of application; if there are several applications, they will have their separate sections.
• streams - list of streams and their parameters
• strm - stream name
• publish_time - The value returned generally represents the number of seconds since 00:00 hours, Jan 1, 1970 UTC
• bandwidth - encoding bandwidth
• resolution - video resolution
• vcoded - video codec spec
• acodec - audio code spec
• publisher_ip -  publisher's source IP  in case RTMP/RTSP/SRT streams are pushed into Nimble instance
• publisher_port - publisher's source PORT  in case RTMP/RTSP streams are pushed into  Nimble instance

Request example
http://127.0.0.1:8082/manage/live_streams_status

Response example

 [

  {

    "app": "larix",

    "streams": [

      {

        "strm": "srt",

        "publish_time": "1706701873",

        "bandwidth": "0",

        "resolution": "1280x720",

        "vcodec": "avc1.42001f",

        "protocol": "MPEGTS",

        "publisher_ip": "192.168.0.95"

      }

    ]

  },

  {

    "app": "rtmp",

    "streams": [

      {

        "strm": "stream",

        "publish_time": "1706701540",

        "bandwidth": "11711170",

        "resolution": "1920x1080",

        "vcodec": "avc1.64002a",

        "acodec": "mp4a.40.2",

        "protocol": "RTMP",

        "publisher_ip": "192.168.0.95",

        "publisher_port": 60349

      }

    ]

  }

]

Get RTMP settings

Request URL
/manage/rtmp_settings

Response parameters
They are nested under RtmpSettings node.

• hash - response hash
• interfaces - which interfaces are set to process RTMP streaming
• login - login for RTMP publishing
• password - password for RTMP publishing
• duration - default chunk duration
• protocols - which protocols' streams are generated as output
• apps - specific applications' settings
• abr - individual ABR streams settings
• app - application name
• stream - stream name
• streams - single bitrate streams included into the ABR, each having its app and stream names

Request example
http://127.0.0.1:8082/manage/rtmp_settings

Response example
{"RtmpSettings":{"hash":"1414983917310","interfaces":[{"ip":"*","port":1936}],"login":"","password":"","duration":6,"protocols":["HLS"],"apps":[],"abr":[{"app":"nimble_live_abr","stream":"abrstream","streams":[{"app":"live","stream":"stream"}]}]}}

Get MPEG-TS connections status 

Request URL
/manage/mpeg2ts_status

Response parameters
They are nested under Cameras node. Each stream has entries for

• id - stream ID
• st - the stream status
• tr - processed traffic delta
• bw - current bandwidth 
• pmts - processes details

Request example
http://127.0.0.1:8082/manage/mpeg2ts_status

Response example
{"Cameras":[{"id":"5456fab17d5c00547f000002","st":"online","tr":"0","bw":"440623","pmts":[{"id":4096,"pids":[{"pid":256,"t":27},{"pid":257,"t":15}]}]}]}

Get MPEG-TS settings

Request URL
/manage/mpeg2ts_settings

Response parameters

• CamerasHash - response hash
• Cameras - information about each stream
• id - source stream ID
• ip - IP of the source stream
• port - source stream port
• protocol - whether UDP or HTTP is used.

Request example
http://127.0.0.1:8082/manage/mpeg2ts_settings

Response example
{"CamerasHash":"1414986417897","Cameras":[{"id":"5456fab17d5c00547f000002","ip":"127.0.0.1","port":1234,"protocol":"udp"}]}

This set of APIs is the first step to building reliable and robust load balancing. If you have any questions or further use cases which you'd like to cover, let us know about it, any feedback is appreciated. Check also a full Nimble Streamer API reference.

Update: You may also find useful the Nimble Streamer routes control API and RTMP streaming API which allows controlling Nimble behavior.

Related documentation

Nimble Streamer, Nimble Streamer API reference, Nimble Streamer routes control API, RTMP and MPEG-TS streaming, WMSPanel API reference, RTMP playback and streaming via Nimble,

The State of Streaming Protocols - October 2014

WMSPanel team continues analyzing the state of streaming protocols. October had a lot of connections which means more of interesting content to look at.

We still see the increase of HLS share (now it's 67%). The share of RTMP (23%) and RTSP (9%) has decreased but the absolute number of connections did not go down significantly which means that newcomers preferred HLS. Other protocols did not have significant changes. MPEG-DASH has shown a small growth by several thousands of connections.

All these metrics are calculated based on 870M+ views.

The State of Streaming Protocols October, 2014

Publishing RTMP from Nimble Streamer to Fastly CDN

UPDATE: RTMP publishing is now a deprecated function of Fastly CDN.

Please read RTMP republishing via Nimble Streamer article and RTMP feature set digest page of Nimble Streamer for more information.

RTMP, RTSP and Icecast pulled streams fallback

RTMP to HLS transmuxing is one of the popular use cases of Nimble Streamer as well as other protocols transmuxing. It allows producing HLS with high efficiency and low resources consumption which allows building robust streaming solutions. However, Nimble is just a link in delivery chain and other elements like source transcoder or origin server may go down. This is why a robust live streaming network needs to have multiple media sources. And Nimble Streamer needs to handle all fallback sources and still produce output streams with no interruptions.

This is why Nimble Streamer team introduces RTMP, RTSP and Icecast pull sources fallback support.

When you define RTMP pull streams, you just specify 2 or more sources of the same pulled stream and if the first stream goes down, other stream will be picked up for processing.

To set this up you need to follow basic setup steps described in Pull RTMP to HLS transmuxing article. The main enhancement is multiple edit boxes in Live pull settings for adding new pull URL.

Being in the list of pulled streams, you can click on Add URL and see the following dialog.

Here the URL is a mandatory field while Fallback URLs can be added one by one, regardless of their number. Saving the settings will launch the RTMP to HLS processing. Also you can use advanced RTMP pull settings to specify incoming RTMP streams. Please read the Processing RTMP and RTSP pull streams per request article for details.

WMSPanel also provides multiple edit capabilities. You may edit RTMP pull sources as a text setting to able to do things like:

• adding big amount of streams;
• making backup and restore of settings;
• transition of settings between servers.

Just click on Multiple edit button to use the dialog as shown below. Here, a "dr_urls" is a field containing fallback links mentioned above. "url", "application" and "stream" are the fields available in the "Add URLs" dialog.

With this entire failover feature you can easily handle RTMP disaster recovery in case of any origin source availability problems.
You may also consider using RTMP streaming API to control this behavior remotely.

Take a look at this tutorial showing the setup process for Icecast as an example.

More capabilities 

Also take a look at RTMP setup in Nimble Streamer video tutorial to learn more about basic RTMP scenarios.

If you need to secure your RTMP streams, you can also use SSL for RTMP transmission.
When transmuxing incoming RTMP streams, you can use Nimble Advertizer for server-side ads insertion. It provides a framework for inserting pre-roll and mid-roll ads into live streams for further output via RTMP, SLDP and Icecast with custom business logic and per-user ads.
Visit Advertizer web page to find out more.

If you need to change content parameters, like change the bitrate, use our Live Transcoder for Nimble Streamer to transform. It has high performance and low resource usage.

Please also check other live streaming, audio streaming and streaming infrastructure features available in Nimble Streamer and let us know if you face any questions or issues.

Related documentation

Live Streaming features in Nimble, Live Transcoder for Nimble Streamer, RTMP feature set, Pull RTMP to HLS transmuxing, RTMP republishing via Nimble, Audio streaming features

Players and devices statistics API

WMSPanel gives statistics for media servers which include statistics for players and devices. As our customers need to integrate these stats into their of workflow, our team provides pull API for obtaining the aggregated data.
We introduce a pull API for players and devices metrics to give more flexibility in data analysis.

All you need is to make some setup on WMSPanel side and then make proper API calls using any programming technology you like. Let's go step by step.

1. Set up API on WMSPanel side

To start using the API, you need to make small setup on WMSPanel side.

First, the API should be enabled by WMSPanel team, so please contact us to enable it for your account. Then you need to follow these steps:

1. Go to Control menu -> API setup -> Pull API tab.
2. Copy Client ID for further usage in your scripts.
3. Click on "Generate New Key" to copy API key as well. This will be used for further authorization.
4. Populate white list with IPs of the hosts where you will be making API call from, like your development machine or your web server. Just enter new IP and click on "Add IP to Whitelist" one by one.

Check this screenshot as a reference: