Using SMIL for adaptive bitrate VOD via HLS

Since we've released MP4 to HLS VOD transmuxing, we were asked to add multi-bitrate (adaptive bitrate) support for VOD. Media industry has a proven way to implement this kind of interaction which is called Synchronized Multimedia Integration Language, or SMIL.

Now we have SMIL support for MP4 to HLS transmuxing in Nimble Streamer.

Let's see how it's set up and how a user may stream ABR VOD.

Nimble Streamer setup

First, you need to install Nimble Streamer if you haven't done it yet.

Now, with the content being located in some local directory - e.g. /var/www/video/ - you need to define a streaming route for Nimble Streamer so the user could access the content via HLS or progressive download with appropriate player. Read how to setup VOD streaming using WMSPanel.
You should try playing the uploaded files via HLS before going to the next step to avoid any permissions or other related problems on your server.

Creating SMIL file

Now when Nimble is able to stream your single-bitrate files, let's compose sample SMIL file. For this example we assume you have /var/www/video/ directory with bigbuckbunny_450.mp4 and bigbuckbunny_750.mp4 files.

The following SMIL file should be created to enable adaptive bitrate streaming:

Language channels for ABR HLS

Update: New article about MPEG-TS transmuxing to HLS, RTMP etc is available. Also, read general Nimble Streamer MPEG2TS feature set page.

Selecting interface for incoming requests

Nimble Streamer is getting users' HTTP requests via some designated port using all available interfaces. Port number is specified in configuration file while our customers could not specify which interface Nimble should listen at.

That's why we introduced listen_interfaces configuration parameter. It's specified in in /etc/nimble/nimble.conf file just as any other config parameters. You can set up list of interfaces for Nimble to work with. If you have 2 IP addresses - one for web server and one for streaming media and you want both to work with port 80, then create this parameter in Nimble config and specify an IP which you need to be handled by Nimble Streamer.

By default Nimble Streamer listens to all interfaces which is an equivalent to this:

listen_interfaces = *

If you need Nimble to listen at single IP, specify it like this:

listen_interfaces = 172.16.1.9

Several interfaces are split by commas:

listen_interfaces = 127.0.0.1, 172.16.1.9

Once the parameter is specified, re-start Nimble:

sudo service nimble restart

That's it. Let us know if you have further questions regarding Nimble set up.

Related documentation

Nimble HTTP Streamer, Nimble configs explained, Making robust streaming with Nimble Streamer, Streaming VOD with Nimble Streamer

Geo-location streaming load balancing for Nimble Streamer

Nimble Streamer is typically used for building robust and efficient streaming delivery infrastructure. It's used both as an origin for VOD HLS and live HLS transmission, as well as a light-weight edge re-streaming server. If viewers are distributed across the globe, a streaming provider needs to be able to balance users' requests between various edges.


UPDATE: WMSPanel team released an article describing the recommended approach to load balancing for HTTP-based protocols like HLS, DASH, Icecast etc. Please use the technique described there to build your own balancing solution rather than HTTP redirect-based approach shown below.
We've created a reference balancing code for geo-location and put it under our github account. It's a PHP class with an example of its usage. Please try it for your use case.
The text below is considered obsolete.


Any Nimble Streamer instance can work as a geo-location balancer. So when the viewer's player attempts to connect to get the media stream, Nimble may check its geographical location and redirect client to new location close to client to get the content from.

Let's see an example of small streaming network with 2 active streaming servers at separate continents.

Set of servers to work with.

paris.wmspanel.com is a European location in Paris, France.

• It provides the VOD HLS as origin from local /var/content/ directory via links like http://paris.wmspanel.com:8081/vod/filename.mp4/playlist.m3u8 .
• Also, it has a live ABR HLS stream available via http://paris.wmspanel.com:8081/live/stream/playlist.m3u8.

Check this screenshot to see VOD setup.

Setting up VOD HLS streaming.

Check also UDP MPEG-TS to ABR HLS transmuxing set up.

atlanta.wmspanel.com - North American location in Atlanta, U.S.

• This server makes HLS re-streaming of live stream provided by paris.wmspanel.com. It also uses efficient caching to reduce latency and improve viewing user experience.
• It also provides the VOD HLS from local /var/media/ directory via links like http://paris.wmspanel.com:8081/media/filename.mp4/playlist.m3u8 . It's a copy of VOD content located in Paris.

Set up VOD for U.S. server.

Set up edge live re-streaming for HLS.

For our example, there are users from France, Spain and Portugal who try to get VOD content from Atlanta. They need to be re-directed to Paris server and get content from there. So we need to create 3 geo redirect routes for atlanta.wmspanel.com as follows. Notice that each geo route allows specifying part of URI which you need to be processed so the other URIs won't be affected.

So after setting up 2 VOD routes, 1 live re-streaming routes and 3 geo routes we have the following picture:

Final routes setup.

Now with described configuration this streaming infrastructure may load balance visitors across two continents according to the type of streaming. You may also use DNS failover scheme which is used in this real life streaming example of our demo page streaming.

If you face any complex streaming scenarios, please contact us so we could help you propose the best usage if Nimble Streamer.

Related documentation

Nimble HTTP Streamer, Pay-per-view for Nimble Streamer, MP4 transmuxing to HLS VOD streaming, Streaming VOD with Nimble Streamer, Mixing multiple MPEG-TS sources into ABR HLS, MPEG-TS UDP conversion to HLS with Nimble, Load balancing and failover with Nimble Streamer, HDS re-streaming

Upgrading Wowza 4 with WMSPanel agent

Now when Wowza Streaming Engine 4 is being successfully deployed to our customers' servers, we're often asked about how to migrate WMSPanel agent correctly.

There are 2 options:

1. Install a new instance of WMSPanel agent. In this case all the data from previous instances will be safe and can be accessed via all daily statistics. You just won't be able to browse through old servers which is just a matter of convenience but not data integrity.
2. Make manual migration of existing agent. In the panel, everything will remain the same from end-user point of view.

The migration procedure is easy.

1. Install Wowza 4 and move your applications and settings according to Wowza migration procedure.

2. Copy lib\WowzaAgentInstaller.jar file and lib\wmspanel-lib folder to new server.

3. Copy Server.xml to new server. If Server.xml cannot be copied due to some special settings, copy the following properties:

• PanelUrl
• PanelUuid
• PanelPassword
• and also copy the following server listener:

<ServerListener>
<BaseClass>com.wmspanel.WmsPanelAgent</BaseClass>
</ServerListener>

• If you use WMSAuth hotlinking protection or other paywall features, also copy ServerAuthorizationProperties

If you have any questions or issues, please contact our support team.

Related documentation

Admin panel for Wowza Streaming Engine 4, Admin panel for Wowza 2 and Wowza 3, End user reporting for Wowza, Paywall for Wowza, Geo and IP range restriction for Wowza, Wowza hotlinking re-publishing and re-streaming protection,  

Streaming HLS and MPEG-DASH from behind reverse proxy

There are use cases when a media streamer is putting Nimble Streamer media server behind the reverse proxy. This is usually done for security reasons and works pretty good.

But there is a side effect of this solution. For each connection, Nimble Streamer gets the IP of the proxy instead of IP of the viewer. This means that geo-location reporting for HLS and other HTTP protocols would be shown incorrectly. Also, WMSPanel hotlinking protection and geo-lock will not work since it's based on IP as well.

For HTTP-based protocols like HLS, MPEG-DASH, SmoothStreaming and Progressive download this can be solved via processing X-Forwarded-For header.

We've recently added processing capability option to handle this HTTP header in Nimble Streamer. You need to follow these easy steps.

1. Open Nimble Streamer configuration file called /etc/nimble/nimble.conf and add the following parameter:

cdn_origin = true

2. Restart Nimble.
For Debian/Ubuntu run:

sudo service nimble restart

For Red Hat Enterprise Linux 6 / CentOS 6 run:

sudo service nimble start

That's it. Nimble Streamer will be obtaining customers' IP addresses correctly.

Web interface changes for WMSAuth

If you use WMSAuth paywall feature set and your web interface is also located behind the proxy, (e.g. in case you use CloudFlare) you may also need to update your media signature code, as described in this article.


Contact us if you face any difficulties with the setup.

Related documentation

Nimble Streamer, Paywall for Nimble Streamer, Nimble Streamer HTTP hotlinking protection, 

Improving cache control for VOD re-streaming

One of the most popular use cases of Nimble Streamer is the ability to re-stream VOD transmission from origin servers to viewers. This allows building media content delivery networks. The common VOD scenario looks like this:

1. An edge server is getting request for a file from a media player. This file may be a chunk for HLS or DASH, or progressive download file.
2. This request is made to the origin server and the file is downloaded.
3. As soon as part of the file is available, it's returned to a viewer.
4. When the file is downloaded, it's placed into file cache.
5. If file cache is full, the file is placed into RAM cache.
6. When this file is requested again, it's taken from edge cache rather than from origin server.

This scenario allows media players continue to play video from the latest chunk in case of edge server restarting (due to update procedure or some crash).

The media which is stored on origin may be delivered via HLS, DASH and progressive download. The origin server may be based on any software, including Nimble. Please read VOD HLS streaming real-life example for it.

Here's a brief picture describing the possible VOD delivery network configuration.

Example of Nimble-based progressive download infrastructure.

Caching is an important part of re-streaming process. It allows saving origin resources and bring the content closer to end users. As any cache, Nimble's cache has its life cycle. Let's see how VOD cache works and how you can control it.

Cache is stored on edge server in a designated directory which is set up in the configuration file via "cache_path" variable. Its default value is "/var/cache/nimble".

There are several parameters you can use for controlling cache size.

Max file cache size. In order to avoid the disk space overflow, you can limit the cache size by specifying "Max file cache size (MB)" parameter via web UI. This will limit the space available for cache.

Cache timeout. You may still face a situation when Nimble needs more space than it's been provided.
To avoid this, set "vod_cache_timeout" parameter which contains amount of hours to keep the files and chunks. Its default value is "24" which means a file will be kept for a day and then cleaned up. If you have limited number of files and wish them all be stored on edge servers for as long as possible, you may set it up for some big number like 720 (an average month).

Minimum storage time. You've set cache timeout and you still have a space limitation, however your VOD files may overdue fast. In this case you can use Least Recently Used (LRU) caching mechanism. Nimble discards the least recently used items first using the time from the config. You may set this time via "vod_cache_min_storage_time" parameter. If Nimble needs to store new file or chunk in cache and the cache is full, then the chunks with earlier time of usage will be removed. Use this parameters after you've set "vod_ cache_timeout" described above.

See also: FAQ: How can I clear Nimble Streamer cache?
Nimble native API also has methods for clearing cache for exact items: Get cache key and Delete items from cache.

These settings allow effectively controlling cache behavior and building flexible and reliable delivery networks.

You can also take a look the video where Jan Ozer of Streaming Learning Center and Yury Udovichenko of Softvelum discuss pros and cons of dynamic packetizing of live, VOD and DVR content. It gives more details about Nimble Streamer VOD streaming.

Related documentation

VOD streaming features of Nimble Streamer, MP4 transmuxing to HLS VOD streaming, Streaming VOD with Nimble Streamer, Nimble Streamer configuration

Wowza statistics behind reverse proxy

There are use cases when a media streamer is putting Wowza Streaming Engine behind the reverse proxy. This is usually done for security reasons. This usually works good.

But there is a side effect of this solution. For each connection, Wowza gets the IP of the proxy instead of IP of the viewer. This means that geo-location reporting for Wowza would be shown incorrectly. Also, WMSPanel hotlinking protection for Wowza streams will not work since its hash is based on IP as well.

For HTTP-based protocols like HLS, MPEG-DASH, HDS and SMOOTH this can be solved via processing X-Forwarded-For header which Wowza provides.

We've recently added processing capability option to WMSPanel agent handle this HTTP header. You need to follow these easy steps.

1. Open Server.xml file and add the following property.

<Properties>
    <Property>
        <Name>PanelCdnOrigin</Name>
        <Value>true</Value>
    </Property>
</Properties>

2. Restart Wowza.

That's it. WMSPanel agent will be obtaining customers' IP addresses correctly.

Please notice that HTTP header is available in Wowza versions 3.x and later. Contact us if you face any difficulties with the setup.

Read more about WMSPanel streaming reporting.

Related documentation

WMSPanel, Control panel for Wowza Streaming Engine 4, End user reporting for Wowza, WMSPanel examples and case study, Paywall for Wowza, Dispersa streams monitoring, 

Mixing multiple sources for adaptive bitrate HLS

We've recently introduced new Nimble Streamer feature set for converting UDP MPEG-TS streams (muticast, broadcast and unicast) into adaptive bitrate HLS streams. This functionality allows covering more live streaming use cases with the same light-weight HLS server.

Update: New article about MPEG-TS transmuxing to HLS, RTMP etc is available. Also, read general Nimble Streamer MPEG2TS feature set page.

The simple and traditional workflow is when a single camera or transcoder pushes the UDP MPEG-TS video+audio stream to Nimble Streamer origin server. This origin transforms incoming stream to HLS which is consumed by viewers. So whatever is pushed to Nimble, is transmuxed to HLS.
The same workflow is used for transmuxing MPEG2TS over HTTP to HLS.

However this is not the only streaming scenario possible with ABR Nimble feature set.

Mixing streams

Delivery network example.

Let's take a look at example on a picture above. You may see 2 cameras which push data to a transcoder. This transcoder produces unicast MPEG-TS streams and transmit them to Nimble Streamer origin for further transmuxing to HLS. From other end, viewers connect to Nimble edge servers, and those servers request data from Nimble origin.

The best thing about this is that you can mix any of those MPEG-TS sources' streams into single multi-bitrate HLS. Let's go deeper into those sample sources' structure.

• Source 1 contains 2Mbps H264 video stream (Video 1.1) and 256Kbps MPEG-2 AAC audio stream (Audio 1.1)
• Source 2 contains 1Mbps H264 video only stream (Video 2.1) 

We need to get an output of an ABR HLS stream which contains the 2 outgoing streams with 3 media components:

1. Video 1.1+Audio 1.1
2. Video 2.1+Audio 1.1

Transmuxing 3 media streams into ABR HLS.

Media server performance metrics update

We've recently updated servers list available under Servers menu. Previously there were columns for country, server name, IP addresses list, current state, WMSPanel software version and CPU+RAM statistics.

Now we also have a snapshot of connections count and bandwidth for each server. These numbers, as well as CPU and RAM metrics, are updated each 30 seconds so you can see each server performance snapshot in real-time mode.

The snapshot is available for both Nimble Streamer and Wowza Streaming Engine.

Take a look at the following screenshot.

Real-life example of servers list with Nimble Streamer edges and Wowza origin.

You can see:

• Nimble Streamer edges processing several dozens of viewers with 1% of its CPU capacity.
• Wowza origin, processing requests from edge servers as well as from end users.

So you may consider changing the configuration of your streaming infrastructure, like re-routing more viewers on Nimble edges.

If you think about any other useful info which might be useful in this server list, feel free to contact us and describe your vision.

Related documentation

Nimble HTTP Streamer, Nimble Streamer overview video, The Paranoid’s Guide to Internet Video Streaming, WMSPanel YouTube channel, Server tasks remote management via web UI