July 30, 2013

Nimble HLS and Smooth RAM caching for effective re-streaming

Nimble HTTP Streamer was initially designed to cover two basic cases: serve progressive download and re-stream HTTP media protocols, having HLS and Smooth protocols to support first.

Effective HLS and Smooth re-streaming requires effective caching. That's because edge servers may be used in a distributed network for quick and cost-effective delivery for end-users. You may have single Wowza origin to make perfect transcoding and transmuxing and then have Nimble servers to take streams from Wowza, cache them and distribute to users in some geographical or network area.

Nimble Streamer edge for Wowza Media Server origin.

The more effective cache Nimble has, the less expensive is the cost of stream delivery to multiple viewers. There are several basic caching features that help making it better.

For live streaming, each chunk is primarily stored in RAM. This allows a very fast access to the cached content. You can control cache size by using max_cache_size parameter in Nimble config.

Each chunk is stored some limited period of time. Currently it's 30 seconds by default. This time is counted since the last access to the chunk. So the most popular chunks will be stored in RAM allowing quick and easy access.

If you use HLS or Smooth for live streaming delivery, the Streamer handles it properly too. If thousands of viewers are requesting the same new chunk from the stream playlist, Nimble requests this chunk just once and any viewer that is asking for the same, will just get it right after it is downloaded from the origin.

These caching features, along with HTTP streaming load balancing capabilities, allow building effective delivery networks for both live and on demand HLS streaming as well as Smooth streaming.



You can check other Nimble Streamer use cases at the website and contact us in case of questions or suggestions. We have a lot of features in our roadmap so your feedback help us prioritizing them.

Related documentation

Nimble HTTP Streamer,  MPEG-TS UDP conversion to HLS with NimbleMP4 transmuxing to HLS VOD streamingNimble configs explainedHLS re-streaming set up, Smooth Streaming supportGeo load balancing with Nimble StreamerHTTP pseudo-streaming progressive downloadProgressive download re-streaming, Running Nimble on privileged portsRAM caching for effective re-streamingNimble disk cache usageNimble Streamer HTTP hotlinking protectionNimble Streamer geo-location restrictionWMSPanel Slideshare

July 29, 2013

WMSPanel products for education

Education is a big area of online media industry. Universities and colleges provide video to improve educational process. We get more customers from this field so let's see how WMSPanel and its features and products help educational organizations.

Discount and yearly payment


Geo-location reporting usually requires additional monthly fee. For education, it's free. This gives a 20% discount in compare to basic price. In addition, a yearly subscription is also available because accounting departments prefer having single payment rather than periodical. So having a discount, the yearly price starts with 240 USD per year.

Camera streaming and recording control


Recording or providing live cast of lectures requires the ability to control streaming process. This is where server management and MediaCaster and LiveStreamRecord streams control capabilities can help. You can set up equipment, set up Wowza setting and set up streams control to allow start/stop/reset cameras streams as well as start and stop stream recording to files.

This may be done not only by some IT person but also by any teacher because WMSPanel allows delegate streams control to non-admin users. So the lecturer can start camera stream, start recording to some central storage, then stop them when the lecture is over.

Controlling access to the media


If your education process is tightly integrated with video, you need to know who watched some course and make sure you have the right personal metrics that are not mixed up with other people.

That's where Pay-Per-View framework and WMSAuth protection technologies can help. You can integrate your student management system with your video delivery via pay-per-view which provides statistics for every unique viewer (which basically is a student in our case). So every student may have the courses metrics. Since PPV is based on WMSAuth, the links to the video cannot be re-streamed so you can be sure it's truly unique metrics.

Statistics


Of course, if you provide video for your lectures and seminars, you want to know how popular it is. The entire statistics feature set of WMSPanel work perfectly here. Daily statistics, high precision charts, even real time reporting - everything is applicable here.

To track separate courses you can use data slicing. That might be either slices based on Wowza applications or stream-based slices.

Of course you can also use WMSPanel as a white label to have your panel's look and feel like your own system.

Streaming infrastructure


Having recorded media files you need to make them available to the audience. There are two options here:

  • deliver them via progressive download;
  • make it available via adaptive streaming using HLS for delivery.

Both options are covered by Nimble Streamer.

It serves progressive download as both origin and edge server so you can set up some HTML5 or Flash player on your web page, then set up light-weight Nimble. That's it, the video is accessible.

It can also work as an edge for HLS streaming. So you may have Wowza Media Server preparing media for adaptive streaming and then deliver it via the network of small Streamers that will cache your data and deliver to multiple viewers. This is especially important for universities which open there courses online for free. So you set up a "multi-tool" Wowza as origin, then bring up any number of Nimble edges and your're ready to deliver.

Of course the same applies to live lectures which you might deliver to the audience sitting both within campus and outside of it. HLS works perfectly for live web casting so the same set up is good for it as well.

Panel accessibility from your network


Some organizations have strict network rules so people ask about internal on-site installations of WMSPanel. We do not provide custom deployments but the service may still be working fine. Just open SSL port 443 for outgoing connections to our service IP addresses (here's more than one IP as we support DNS failover). That's the only thing that is needed for our service to work with your infrastructure.

Now sign up to see our service in action or contact us to learn more about your particular case.

Read more use cases in our Case study section.

Related documentation


End user reporting for WowzaStatistics for CDN usersDaily statisticsRestriction solution for WowzaWowza hotlinking re-publishing and re-streaming protectionPay-per-view for Wowza Media ServerData slices for statisticsStreamed slices for WowzaWhite label branded panelServer management for WowzaMediaCaster and LiveStreamRecord streams controlNimble HTTP StreamerWMSPanel Slideshare

UPDATE: High precision report for Wowza

We've recently released the update for high precision reporting functionality.
It now has a chart! By default it shows last 24 hours but may be extended up to 48 hours and it has 5 minutes granularity.

High precision chart.

Please notice that high precision reporting is available via additional subscription.

Related documentation


End user reporting for Wowza Data slices for statistics Streamed slices for Wowza High precision reporting

July 25, 2013

Pay-per-view for Wowza Media Server

WMSPanel uses "agent approach" for accessing Wowza Media Server. This allows making a lot of advanced server control features. WMSPanel is already capable of server managementstreams control and recording and transcoder control. Another great feature set called WMSAuth allows restricting connections by various criteria and protects against hot-linking re-streaming.

A lot of our customers requested upgrading restriction capabilities into a per-user control feature set. So after a while we designed and implemented this approach. We call this a pay-per-view (PPV) feature set.

What are the use cases for this kind of functionality?

  • pay per view logic for streaming media providers as each viewer info has the fact of viewing;
  • pay per minute logic as it allows gathering view time per viewer;
  • re-streaming protection when 1 person pays for 1 account and provides access for group of people;
  • unique visitors statistics calculation (both customer-defined and WMSPanel-based);
  • education video management for per-student metrics;
  • you name it.

Let's see how it works and how you can use it.

Basic business logic


The main workflow is based on push API. WMSPanel agent sends (pushes) user data to customer, then customer decides what to do with each individual unique user and replies with the decision back to the agent. This works as follows:
  1. Customer implements API handler that is able to accept POST requests.
  2. Customer adds PPV info into existing media links.
  3. WMSPanel provides web UI for setting up handler URL and some other parameters.
  4. WMSPanel agent which works on Wowza server side sends request to handler on a periodical basis which is defined via web UI. Sync contains all users' IDs, IPs, view time and media names.
  5. Customer handler makes required decisions based on company business logic and sends response to agent. This response specifies which users must be blocked for watching media.
  6. Blocked users are shut down immediately, while others keep watching.
  7. WMSPanel agent may be installed on any number of Wowza servers. All agents will send sync-ups to the same single API handler.
WMSPanel pay-per-view for Wowza Media Server.
Here's the flow chart showing the basic scenario.

Pay-per-view framework basic workflow.
Let's go step by step. Please open WMSPanel API samples github repo to refer to the code. Please also read WMSAuth documentation because PPV shares the same URL modification and handling mechanics. Now follow these steps.

July 8, 2013

Nimble configs explained


UPDATE: General server setting file description was moved to this article.

Our team develops Nimble Streamer to be available free of charge.

This means that anyone may install Nimble Streamer from our repository and keep using it manually, without our service. Of course, the best user experience is achieved via our excellent WMSPanel reporting and control panel but it's up to a user to decide whether he wants to pay for comfort or he needs more financial flexibility.

The key element of "non-service" usage of Nimble Streamer is the ability to configure the server manually via configuration files. To change the Nimble behavior, you need to change the configs. WMSPanel does this remotely, you can do it yourself. Knowing configs structure allows doing manual changes without WMSPanel. This also brings an ability to make custom automated configuration of server network in case of complex deployments.

So let's see configuration files structure.

General settings


Server general settings are stored in /etc/nimble/nimble.conf file.

Read this config file description to learn more about how to control Nimble behavior and attributes.

Also, take a look at performance tuning guide to see how much your hardware can handle with best settings of Nimble Streamer.

Routes settings


Routes settings are stored in /etc/nimble/rules.conf.

UPDATE: This description is outdated. We recommend using WMSPanel for primary setup in order to see how parameters and their values are mentioned in config.

The typical routes settings config is listed below. It's a JSON-formatted data set. The example below is formatted for educational purposes, the real config is presented as a single line.
/* please do not modify this file, all changes will be overwritten by wmspanel agent */
{"SyncResponse":
  {
    "status":"success",
    "RoutesHash":1373269971,
    "Users":0,
    "Routes":
    [
      ["","/local/raw/","file:///var/www/content/raw/",""],
      ["streamingtestingserver.com:8080","/live/", "live.nimblestreamer.com:8080","/"],
      ["GEO_R_AQ","","live2.nimblestreamer.com",""],
      ["GEO_R_IP_R_Range 2","","test.nimblestreamer.com",""]
    ],
    "IpRanges":
    [
      ["IP_R_Range 1","192.168.0.0/22"],
      ["IP_R_Range 2","172.168.0.0/24"],
      ["IP_R_LocalHostRange","127.0.0.0/8,128.0.0.0/8"],
      ["IP_R_all","0.0.0.0/0"]
    ]
  }
}

The first comment line warns you that routes settings are replaced after the next time you change settings within the panel. If you don't want the settings to be replaced, just remove the server from the panel or make its backup.

First 3 parameters - status, routeHash and Users - are used for syncing purposes so you don't need to change them.

Next sections is Routes, it contains the routes defined within a panel. Each entry separated by [ and ] is a set of 4 strings. They are Domain From, Path From, Domain To and Path To respectively.

For progressive download or HLS re-streaming set up those 4 elements work straight-forward, please read this or this post for set up details.

For geo forwarding see route 3. The first element stands for country of original request made by "GEO_R_" prefix and ISO country codes. In this example we see "GEO_R_AQ" having "AQ" country code meaning Afganistan. Path From and Path To are blank, with only Domain To pointing to the domain and port of destination of request.

For IP range forwarding you can check route 4. It has the same "GEO_R_" prefix followed by the name of IP range. All names of IP ranges are defined within WMSPanel and synced in this config as well. Let's take a look at IpRanges section. Each entry has its name used in WMSPanel and in the routes above. It also has CIDR defining the range itself. So having list of ranges you can use them in any of your routes.

Those are all parameters you may need for controlling server behavior. Once you make a change please re-start Nimble by running
sudo service nimble restart

Nimble Streamer feature set will be extended further and its configuration files will also be changing. Our team will be updating this description to keep it up-to-date with latest releases.

You can try Nimble Streamer any time and contact us in case of any questions. We're looking forward to your questions and suggestions.

Nimble Streamer automation example


Let's see how to automate Nimble installation and set up in 2 easy steps. Let's take Ubuntu as example while you can use a variety of other Linux distributions.

1. Install Nimble Streamer


Add new repository for Nimble Streamer package installation and update repositories:
echo "deb http://nimblestreamer.com/ubuntu precise/" >> /etc/apt/sources.list
apt-get update
apt-get install nimble

2. Configure Nimble Streamer


Create the content directory and point Nimble to take video from there for VOD streaming:
mkdir -p /var/www/video/
echo ‘{"SyncResponse":{"Routes":[["","/vod/","file:///var/www/video/",""]]}}’ > /etc/nimble/rules.conf
service nimble restart
The media is now available at http://<server ip>:8081/vod/ and you can stream it via HLS with URL like this:
http://127.0.0.1:8081/vod/sample.mp4/playlist.m3u8

Please also take a look at Nimble Streamer presentation with server functionality overview and link in "Related documentation" section below.

You can read "The Paranoid’s Guide to Internet Video Streaming" by Thomas Gires to see real-life example of using pay-per-view feature set along with Nimble Streamer as light-weight edge for HLS re-streaming.

You may also consider WMSPanel server tasks remote management via web console for convenient control of server-side commands. Launch any command, like ffmpeg process, with no need to log in via SSH, just an easy-to-use web interface.


Related documentation


Nimble Streamer, Live Streaming via Nimble, Live Transcoder

July 4, 2013

Stream-based data slices

We introduced data slices concept in late 2011 and changed it to properly cover Wowza Media Server in early 2012. It was based on applications - you can define regular expression for application names and get the date gathered for specified app only. Application is the basic way to separate different streaming instances within Wowza, like different clients which have different VOD content or live streams. So each slice was a reflection of one or more applications and all the data we have for daily stats - like connections, traffic, bandwidth, geo-location, duration, billing - was collected using this basis.

Right after we launched slices, people started asking if we plan making data slices based on stream names. Many people want to know the performance of individual streams rather than entire application. Streams reports were not enough in some cases. So we've been gathering this feedback and changing our infrastructure and processing algorithms to cover this upcoming approach.

Now we introduce new stream-based data slices. This means that you can now create 2 kinds of slices - application-based and stream-based. Application slices has advanced include/exclude rules distinction while streams are a bit more simple. Let's see what it can do.

Slices management links


To create a slice you need to click on data slices Manage link and then click on Create streamed data slice. You will see a dialog asking for Slices short name and description. Enter them and click on Create slice button. You may also specify time zone.

Creating a slice.
You see 4 sections:
  • Slice information - you can change these detailed settings by clicking on Edit link on top right corner.
  • Assigned servers.
  • Rule for defining streams names.
  • Assigned users.

Slice information contain the following settings:
  • Name and description - which you previously entered to identify the slice.
  • Show servers details for users defines if end-users (who we will assign to this slice later) are allowed to see details about servers which send their data to this slice.
  • Billing details - they are used in Billing feature set.
Streamed slice basic info. 
Then you need to assign servers to the slice. This is how you define which servers will be sending information to this slice. You need to add them from the list of servers available in the system.

Assigning servers.
The next step is to define a rule with the regular expression for the streams and applications which you would like to track. Each expression may cover any number of streams or apps. So learn more about regexes and set up what you like.

Adding rule for a stream.
Rule is ready.

In case of Wowza you may also specify exact virtual host to collect data for.
For Nimble Streamer you may specify domain name. In both cases you may use regular expressions to describe the match.

Specifying domain for collecting its data.

Then you need to assign new or existing users to the slice. Please read this blog post for detailed description.


So now you're all set. Having this slice you can switch to it and see what's going on with your stream. You can set up white label branded panels and create accounts for your clients so they could see these statistics too. All end-user experience related features can be used with streamed slices.

Notice that streamed slices are eligible for subscription payments. Please refer to our full prices page for details. We recommend using lite slices as their reduced functionality is mostly covered with streamed slices capabilities hence they are half the price from the original full slices. Please contact sales for enabling this pricing option.

To see this in action you need to make a sign up to try our panel.

Read more about slice-related features in on the dedicated Slices & Branding page.

Read more about WMSPanel streaming reporting.

Related documentation


End user reporting for WowzaDaily statisticsHigh precision reportingGeo location reportingData slices for statisticsBilling customers using daily statsWhite label branded panelScreencast for daily statisticsScreencast for data slices and white labelStatistics import API


July 2, 2013

Wowza and Nimble Streamer daily statistics reporting API

Many customers say that WMSPanel provides excellent reporting. That's true - we get as much information as we can from Wowza, combining it in the proper way and show via nice web interface available on any device.

Knowing that we have a lot of useful information for all metrics people want to get this data for further analysis within their own environment. Building custom stakeholder reports or charging clients using third-party billing - this requires raw data delivered on a regular basis. Of course we have CSV import for most of our reports, but many people prefer automated data collection.

And so we made a statistics push API for delivering your data to you. Go to Control / API setup menu and check the following form.

Set up Wowza statistics POST requests handler.
It allows setting up periodical export of data into your handler. The export is made each day, for the past day.
It also allows making data export on demand.

Here is how it works.

  1. You create a handler on your web site. This should be a web application that may receive and handle HTTP POST requests.
  2. You set up URL and Token for sending the data. URL must point to a valid handler. Token is not yet used but it will be used in later API versions for secure your data transfer.
  3. Clicking on Save button you will schedule the periodical data export. The data is scheduled to be exported each day after midnight once the system has available resources for it. We guarantee that you get your data for previous day even though we cannot predict the exact time of arrival.
  4. Clicking on Export button you set your data export request into the FIFO queue of manual requests. The system takes requests one by one from the queue every minute and processes them.
  5. Once the system takes any data export request, it makes HTTP POST request and sends it to the handler. The request body contains JSON-formatted daily statistics for each data slice in the customer account. Please find our github-hosted JSON sample.

So having set up the handler, you can be sure you get previous day information next day. If you need specific date range, you click on Export and specify dates. After several minutes you get required info. This is especially useful for testing purposes when you implement and debug your handler.

Set date range for manual export.
Having this feature set you can use WMSPanel Wowza statistics not only via excellent web interface available on any device but also within your own system.

You can also use streams publish/un-publish events notifications API in case you need to know when your stream goes down.

Please also check full WMSPanel API reference for other API methods.

Contact us if you have any specific questions or if you have any other feature request for our API. We're opened for any feedback and we plan extending this functionality.

Please also check full WMSPanel API reference for other API methods.
This statistics export is also available for Nimble Streamer, the light-weight HTTP streaming server for HLS, Smooth and progressive download. WMSPanel is an official GUI for Nimble Streamer.

Read more about WMSPanel streaming reporting.


Related documentation


WMSPanel API referenceReal-time stats pull APIDaily stats APIDuration stats APIPlayers and devices stats APIDeep stats pull APIEnd user reporting for WowzaStatistics for CDN usersDaily statisticsDevices and Players report for WowzaData slices for statisticsStreamed slices for WowzaBilling customers using daily statsNotifying on RTMP stream (un)publishPaywall for WowzaNimble Streamer