Nimble HLS and Smooth RAM caching for effective re-streaming

Nimble Streamer was initially designed to cover two basic cases: serve progressive download and re-stream HTTP media protocols, having HLS and Smooth protocols to support first.

Effective HLS and Smooth re-streaming requires effective caching. That's because edge servers may be used in a distributed network for quick and cost-effective delivery for end-users. You may have single Wowza origin to make perfect transcoding and transmuxing and then have Nimble servers to take streams from Wowza, cache them and distribute to users in some geographical or network area.

The more effective cache Nimble has, the less expensive is the cost of stream delivery to multiple viewers. There are several basic caching features that help making it better.

For live streaming, each chunk is primarily stored in RAM. This allows a very fast access to the cached content. You can control cache size by using max_cache_size parameter in Nimble config.

Each chunk is stored some limited period of time. Currently it's 30 seconds by default. This time is counted since the last access to the chunk. So the most popular chunks will be stored in RAM allowing quick and easy access.

If you use HLS or Smooth for live streaming delivery, the Streamer handles it properly too. If thousands of viewers are requesting the same new chunk from the stream playlist, Nimble requests this chunk just once and any viewer that is asking for the same, will just get it right after it is downloaded from the origin.

It’s important to notice that caching for HLS re-streaming is not available when HTTP Origin mode is enabled. In HTTP origin mode, Nimble Streamer works as a pass-through, fetching content directly from the Origin server without caching it at Edge server.

Related documentation

Nimble Streamer, HLS re-streaming set up, RAM caching for effective re-streaming, Nimble disk cache usage,

WMSPanel products for education

This page is outdated. Please visit our Snapshots page for more details on our products' usage.

UPDATE: High precision report for Wowza

Please read the updated version here: High precision report.

Pay-per-view for Wowza Media Server

This article is out-of-date. We've moved its content to our website:
Pay-per-view framework for Wowza Streaming Engine

Nimble configs explained

UPDATE: General server setting file description is located here.

Our team develops Nimble Streamer to be available free of charge.

This means that anyone may install Nimble Streamer from our repository and keep using it manually, without our service. Of course, the best user experience is achieved via our excellent WMSPanel reporting and control panel but it's up to a user to decide whether he wants to pay for comfort or he needs more financial flexibility.

The key element of "non-service" usage of Nimble Streamer is the ability to configure the server manually via configuration files. To change the Nimble behavior, you need to change the configs. WMSPanel does this remotely, you can do it yourself. Knowing configs structure allows doing manual changes without WMSPanel. This also brings an ability to make custom automated configuration of server network in case of complex deployments.

So let's see configuration files structure.

General settings

Server general settings are stored in /etc/nimble/nimble.conf file.

Read this config file description to learn more about how to control Nimble behavior and attributes.

Also, take a look at performance tuning guide to see how much your hardware can handle with best settings of Nimble Streamer.

Routes settings

Routes settings are stored in /etc/nimble/rules.conf.

UPDATE: The routes settings description below is outdated. We recommend using WMSPanel for primary setup in order to see how parameters and their values are set in config. Having the example of config you may continue with controlling Nimble manually.

The typical routes settings config is listed below. It's a JSON-formatted data set. The example below is formatted for educational purposes, the real config is presented as a single line.

/* please do not modify this file, all changes will be overwritten by wmspanel agent */
{"SyncResponse":
  {
    "status":"success",
    "RoutesHash":1373269971,
    "Users":0,
    "Routes":
    [
      ["","/local/raw/","file:///var/www/content/raw/",""],
      ["streamingtestingserver.com:8080","/live/", "live.nimblestreamer.com:8080","/"],
      ["GEO_R_AQ","","live2.nimblestreamer.com",""],
      ["GEO_R_IP_R_Range 2","","test.nimblestreamer.com",""]
    ],
    "IpRanges":
    [
      ["IP_R_Range 1","192.168.0.0/22"],
      ["IP_R_Range 2","172.168.0.0/24"],
      ["IP_R_LocalHostRange","127.0.0.0/8,128.0.0.0/8"],
      ["IP_R_all","0.0.0.0/0"]
    ]
  }
}

The first comment line warns you that routes settings are replaced after the next time you change settings within the panel. If you don't want the settings to be replaced, just remove the server from the panel or make its backup.

First 3 parameters - status, routeHash and Users - are used for syncing purposes so you don't need to change them.

Next sections is Routes, it contains the routes defined within a panel. Each entry separated by [ and ] is a set of 4 strings. They are Domain From, Path From, Domain To and Path To respectively.

For progressive download or HLS re-streaming set up those 4 elements work straight-forward, please read this or this post for set up details.

For geo forwarding see route 3. The first element stands for country of original request made by "GEO_R_" prefix and ISO country codes. In this example we see "GEO_R_AQ" having "AQ" country code meaning Afganistan. Path From and Path To are blank, with only Domain To pointing to the domain and port of destination of request.

For IP range forwarding you can check route 4. It has the same "GEO_R_" prefix followed by the name of IP range. All names of IP ranges are defined within WMSPanel and synced in this config as well. Let's take a look at IpRanges section. Each entry has its name used in WMSPanel and in the routes above. It also has CIDR defining the range itself. So having list of ranges you can use them in any of your routes.

Those are all parameters you may need for controlling server behavior. Once you make a change please re-start Nimble by running

sudo service nimble restart

Nimble Streamer feature set will be extended further and its configuration files will also be changing. Our team will be updating this description to keep it up-to-date with latest releases.

You can try Nimble Streamer any time and contact us in case of any questions. We're looking forward to your questions and suggestions.

Nimble Streamer automation example

Let's see how to automate Nimble installation and set up in 2 easy steps. Let's take Ubuntu as example while you can use a variety of other Linux distributions.

1. Install Nimble Streamer

Add new repository for Nimble Streamer package installation and update repositories:

echo "deb http://nimblestreamer.com/ubuntu precise/" >> /etc/apt/sources.list
apt-get update
apt-get install nimble

2. Configure Nimble Streamer

Create the content directory and point Nimble to take video from there for VOD streaming:

mkdir -p /var/www/video/
echo ‘{"SyncResponse":{"Routes":[["","/vod/","file:///var/www/video/",""]]}}’ > /etc/nimble/rules.conf
service nimble restart

The media is now available at http://<server ip>:8081/vod/ and you can stream it via HLS with URL like this:

http://127.0.0.1:8081/vod/sample.mp4/playlist.m3u8

Related documentation

Nimble Streamer, Live Streaming via Nimble, Live Transcoder

Stream-based data slices

We introduced data slices concept in late 2011 and changed it to properly cover Wowza Media Server in early 2012. It was based on applications - you can define regular expression for application names and get the date gathered for specified app only. Application is the basic way to separate different streaming instances within Wowza, like different clients which have different VOD content or live streams. So each slice was a reflection of one or more applications and all the data we have for daily stats - like connections, traffic, bandwidth, geo-location, duration, billing - was collected using this basis.

Right after we launched slices, people started asking if we plan making data slices based on stream names. Many people want to know the performance of individual streams rather than entire application. Streams reports were not enough in some cases. So we've been gathering this feedback and changing our infrastructure and processing algorithms to cover this upcoming approach.

Now we introduce new stream-based data slices. This means that you can now create 2 kinds of slices - application-based and stream-based. Application slices has advanced include/exclude rules distinction while streams are a bit more simple. Let's see what it can do.

Slices management links

To create a slice you need to click on data slices Manage link and then click on Create streamed data slice. You will see a dialog asking for Slices short name and description. Enter them and click on Create slice button. You may also specify time zone.

Creating a slice.

You see 4 sections:

• Slice information - you can change these detailed settings by clicking on Edit link on top right corner.
• Assigned servers.
• Rule for defining streams names.
• Assigned users.

Slice information contain the following settings:

• Name and description - which you previously entered to identify the slice.
• Show servers details for users defines if end-users (who we will assign to this slice later) are allowed to see details about servers which send their data to this slice.
• Billing details - they are used in Billing feature set.

Streamed slice basic info. 

Then you need to assign servers to the slice. This is how you define which servers will be sending information to this slice. You need to add them from the list of servers available in the system.

Assigning servers.

The next step is to define a rule with the regular expression for the streams and applications which you would like to track. Each expression may cover any number of streams or apps. So learn more about regexes and set up what you like.

Adding rule for a stream.

Rule is ready.

In case of Wowza you may also specify exact virtual host to collect data for.
For Nimble Streamer you may specify domain name. In both cases you may use regular expressions to describe the match.

Specifying domain for collecting its data.

Then you need to assign new or existing users to the slice. Please read this blog post for detailed description.

So now you're all set. Having this slice you can switch to it and see what's going on with your stream. You can set up white label branded panels and create accounts for your clients so they could see these statistics too. All end-user experience related features can be used with streamed slices.

Data slice also provides a mechanism for assigning non-admin users to control certain features of Nimble Streamer. Read this page for more details.

Notice that streamed slices are eligible for subscription payments. Please refer to our full prices page for details. We recommend using lite slices as their reduced functionality is mostly covered with streamed slices capabilities hence they are half the price from the original full slices. Please contact sales for enabling this pricing option.

To see this in action you need to make a sign up to try our panel.

Read more about slice-related features in on the dedicated Slices & Branding page.

Read more about WMSPanel streaming reporting.

Related documentation

End user reporting for Wowza, Daily statistics, High precision reporting, Geo location reporting, Data slices for statistics, Billing customers using daily stats, White label branded panel, Screencast for daily statistics, Screencast for data slices and white label, Statistics import API

Wowza and Nimble Streamer daily statistics reporting API

Many customers say that WMSPanel provides excellent reporting. That's true - we get as much information as we can from Wowza, combining it in the proper way and show via nice web interface available on any device.

Knowing that we have a lot of useful information for all metrics people want to get this data for further analysis within their own environment. Building custom stakeholder reports or charging clients using third-party billing - this requires raw data delivered on a regular basis. Of course we have CSV import for most of our reports, but many people prefer automated data collection.

And so we made a statistics push API for delivering your data to you. Go to Control / API setup menu and check the following form.

Set up Wowza statistics POST requests handler.

It allows setting up periodical export of data into your handler. The export is made each day, for the past day.
It also allows making data export on demand.

Here is how it works.

1. You create a handler on your web site. This should be a web application that may receive and handle HTTP POST requests.
2. You set up URL and Token for sending the data. URL must point to a valid handler. Token is not yet used but it will be used in later API versions for secure your data transfer.
3. Clicking on Save button you will schedule the periodical data export. The data is scheduled to be exported each day after midnight once the system has available resources for it. We guarantee that you get your data for previous day even though we cannot predict the exact time of arrival.
4. Clicking on Export button you set your data export request into the FIFO queue of manual requests. The system takes requests one by one from the queue every minute and processes them.
5. Once the system takes any data export request, it makes HTTP POST request and sends it to the handler. The request body contains JSON-formatted daily statistics for each data slice in the customer account. Please find our github-hosted JSON sample.

So having set up the handler, you can be sure you get previous day information next day. If you need specific date range, you click on Export and specify dates. After several minutes you get required info. This is especially useful for testing purposes when you implement and debug your handler.

Set date range for manual export.

Having this feature set you can use WMSPanel Wowza statistics not only via excellent web interface available on any device but also within your own system.

You can also use streams publish/un-publish events notifications API in case you need to know when your stream goes down.

Please also check full WMSPanel API reference for other API methods.

Contact us if you have any specific questions or if you have any other feature request for our API. We're opened for any feedback and we plan extending this functionality.

Please also check full WMSPanel API reference for other API methods.
This statistics export is also available for Nimble Streamer, the light-weight HTTP streaming server for HLS, Smooth and progressive download. WMSPanel is an official GUI for Nimble Streamer.

Read more about WMSPanel streaming reporting.

Related documentation

WMSPanel API reference, Real-time stats pull API, Daily stats API, Duration stats API, Players and devices stats API, Deep stats pull API, End user reporting for Wowza, Statistics for CDN users, Daily statistics, Devices and Players report for Wowza, Data slices for statistics, Streamed slices for Wowza, Billing customers using daily stats, Notifying on RTMP stream (un)publish, Paywall for Wowza, Nimble Streamer