The State of Streaming Protocols - July 2015

WMSPanel team continues analyzing the state of streaming protocols.

The metrics calculations are based on nearly 1.4 billion views. The stats are collected from 1800+ media servers (including Wowza, Nimble Streamer and Flussonic).

HLS share is at 69% share with slight decrease of RTMP and Icecast numbers. Check the chart below for more.

The State of Streaming Protocols - July 2015.

You can compare that to June stats below.

Mobile broadcasting SDK

This page is outdated. Please visit Larix mobile streaming SDK page.

Adding preconfigured Nimble Streamer server to WMSPanel

This article is obsolete. Please use this latest deployment automation feature to add pre-configured Nimble Streamer instance in your WMSPanel account.

Advanced MPEG-TS delivery over UDP via Nimble Streamer

There is wide range of tasks associated with streaming of media content in MPEG-TS over UDP in corporate networks. Nimble Streamer is capable of transmitting stream to multicast network, DVR Storage, transcoder or presentation monitor.

In some cases it is necessary to transmit multiple programs in a single MPEG-TS stream. This activity is often related to private networks (hotel, enterprise), when using a hardware decoder that supports only MPEG-TS as input.

The distinction of Nimble Streamer is the ability to transmux up to 256 RTMP, RTSP and MPEG-TS streams into single multi-program MPEG-TS UDP stream. For each incoming stream the unique PMT PID, Video PID and Audio PID values are assigned. You can take RTMP, RTSP and MPEG-TS streams from any sources, transmux them into a single UDP MPEG-TS stream and transmit via multicast.

To use this functionality in your workflow you need to install and configure Nimble Streamer.

Install Nimble Streamer

Sign up for WMSPanel account if you haven't done it yet. Then use this instruction to install Nimble Streamer on your operating system.

Setting up UDP publication

Login to WMSPanel, go to "Nimble Streamer" -> "Live streams settings".

First of all, you should specify Nimble Streamer incoming RTMP, RTSP and MPEG-TS streams by following any instruction which sets up those incoming protocols. Please refer to respective digest pages for details. E.g. read about MPEG-TS incoming streams setup or transmuxing RTMP incoming stream.

Now go to the "UDP streaming" tab to set up outgoing UDP stream.

Push the «Add UDP setting» button and specify required parameters.

First require fields are IP (127.0.0.1 in this case) and Port (e.g. 2015) for UDP stream. You may also specify TTL.

In case you need, you may specify Transport Stream ID and Original Network ID.

By clicking on Bind particular local interface you will open additional fields for Local IP and Local Port to bind.

Next you need to specify incoming streams to transmux to MPEG-TS UDP. Enter the name of Source application name and Source stream name; if you start entering existing streams, the UI will propose the autocomplete with those names. The UI will automatically assign PMT PID, Video PID and Audio PID to each stream.

As you enter new app/stream, the UI will add more lines to add more streams.

Parameters PMT PID, Video PID and Audio PID are set automatically for each stream. It is possible to assign these numbers manually, but you must be aware of allowed values. If any stream stops broadcasting, it is excluded from outgoing UDP stream, while the remaining stream numbers stay the same. If you add another incoming stream to existing streams, it will also get appropriate PIDs automatically. So, this feature keeps you worry-free about the whole UDP streaming gets terminated when one or more incoming streams are removed or added.

SDT checkbox opens additional fields for defining Service Description Table, such as Program Number, Service Type, Provider Name and Service Name.

As it was mentioned before, web interface allows to add up to 256 source streams for further transmuxing into single outgoing MPEG-TS UDP stream. Pragmatic constraints depend on media server performance and bandwidth.

FAQ: how do I produce a stream with multiple audio outputs?

First, define output streams in MPEGTS Out as tracks without a video.

Then create UDP setting as shown above, and make it have same PMT PID for each audio and video track.

Testing the UDP setup

You should already have one or several RTMP, RTSP or MPEG-TS streams. For example, you have three streams: stream_low, stream_mid, stream_high.

As these streams are set up, they can be combined into singe outgoing MPEG-TS UDP stream in the "UDP streaming" tab.

Resulting UDP stream will appear in the list.

To test this UDP streaming, open "MPEGTS In" tab ("MPEGTS In" -> "Add UDP stream")

and then specify your UDP stream as a source.

After adding incoming UDP stream, it should appear in the list:

First you see yellow "pending" box, and then after few seconds it becomes green (online).

In the "MPEGTS Out" tab ("MPEGTS Out" -> "Add outgoing stream")

you can choose separate streams by Video PID and Audio PID from your MPEG-TS UDP stream for further streaming.

You can combine Audio PID and Video PID from different PMT PID. 

Resulting streams will be displayed in the list.

After a few seconds all streams become green (synced).

Now if you go to "Nimble Streamer" -> "Live streams" -> "Outgoing streams", you can see, that all your streams are online.

Some useful parameters

You may use a few parameters in Nimble config to change the behavior of MPEGTS processing.

• mpeg2ts_pat_pmt_msec_interval is a time to retransmit for PAT and PMT tables. By default it's 100 milliseconds.
• mpeg2ts_pcr_retransmit_msec_interval is a time to retransmit PCR. By default it's 20 milliseconds.
• mpeg2ts_sdt_msec_interval is a time to retransmit for SDT tables. By default it's 500 milliseconds.
• mpegts_join_timestamps_enabled - some encoders send their samples split by parts to other samples, and all those parts have the same timestamp. This option allows joining those parts to be processed as one sample. It works on receiving side for AVC and HEVC. It's false by default.
• udp_so_receive_buffer - If you see any image artifacts or video freeze when receiving content through UDP via MPEG-TS or RTSP, Nimble allows increasing UDP buffer by setting this parameter.

Using constant bitrate (CBR) and mux rate

When you stream from Nimble Streamer via UDP, a variable bitrate is used for output by default. However, in some cases of using DVB/ATSC and other hardware, a lot of ETR101290, CC and PCR errors appear on a regular basis making it hard to use it.

That happens because of large spikes of data which can be seen on this screenshot.

However, using constant bitrate (CBR) you can flatten the bandwidth to pictures like this one.

Read Constant bitrate and mux rate in UDP streaming article to find out more about this technique.

Nimble Streamer has PCR accuracy of 100%. While hardware receivers accept up to 500ns PCR drift, Nimble Streamer produces PCR with 0ns drift. Also, PCR interval is <20 ms while 40ms is acceptable by most hardware. All these values were checked using StreamGuru.

Troubleshooting

It you've set up your UDP streaming according the instructions above and still don't get the stream, check the following cases which might help.

1. TTL

Try increasing TTL for your interface. Make sure it's not less then hop count between your source and your server. You can refer to TTL and Hop articles for more details.

2. Interfaces routing

If your server has 2 or more interfaces that work with different networks, you may need to set up correct routing to properly use them. You can refer to this article for some examples.


That's it. For more information about setting up incoming and outgoing HTTP MPEG-TS streams, please refer to "Transmux MPEG-TS into HLS, RTMP, MPEG-DASH and more via Nimble Streamer" article.

If you have any questions about installing and configuring Nimble Streamer, please contact us.

Related documentation

MPEG2TS processing via Nimble, Transmux MPEG-TS into HLS, RTMP, Live Streaming features in Nimble, Live Transcoder for Nimble Streamer, Nimble Streamer control API reference

Streaming VOD from remote HTTP storage via Nimble Streamer

Many broadcasting companies use remote media storage as a convenient way of organizing video-on-demand (VOD) streaming infrastructure. The main advantage of dedicated storage is having all files located in single repository, instead of maintaining multiple copies across all edge servers, and save costs on multiple large hard disk drives. The second advantage is ability to organize centralized backup and failure recovery and convenience of managing content, such as adding, removing or replacing media files. This is why storages like Amazon S3 are extremely popular.

Remote storage support is now available in Nimble Streamer. It works with any server, that supports file access via HTTP protocol with HTTPS as an option. Nimble Streamer support HLS, and MPEG-DASH transmuxing as well as progressive download from remotely stored media content. Advanced caching techniques allows Nimble to effectively stream files, those size exceeds available file system capacity.

Nimble Streamer is extremely lightweight and powerful. You can use any cheap virtual machine to stream thousands of files. All you need to do is specify link to your media on remote server via WMSPanel web interface.

Install Nimble Streamer

Please, see how to install Nimble Streamer for your operating system. Or read this blog article to see the example of Nimble Streamer installation on Ubuntu virtual machine.

Set up streaming routes

For local files (available in file system), please check "Set up streaming routes" section of "How to stream VOD with Nimble Streamer" article from our blog.
For media files stored remotely, you can perform the following steps:

1. Check that your media is available from remote Storage via HTTP (just try to simply download any media file). In this step you should have a media URL like this http://s3-us-west-1.amazonaws.com/wmspanel/bunny.mp4 (you can use any available remote storage).

2. Navigate to "Nimble Streamer" -> "VOD & re-streaming routes" menu section of WMSPanel web interface and click "Add VOD streaming route" link.

3. In the appeared dialog specify path, that is requested by your viewers via public URL and path, that requested content is taken from. Select "http://" or "https://" in dropdown list and enter path to your remote files location.

4. Optionally, specify the AWS access key id and AWS secret access key if your media files are protected by Amazon.

5. Click "OK" button.

Your new route will appear in Nimble routes list.

Now if you request http://localhost:8081/s3/bunny.mp4/playlist.m3u8, then Nimble Streamer takes file http://s3-us-west-1.amazonaws.com/wmspanel/bunny.mp4, transmuxes it, and performs HLS streaming.

This works for HLS and MPEG-DASH by default. If you'd like to use this for progressive download, you need to explicitly enable it in Nimble configuration file.
Open nimble.conf file and add the following parameter.

enable_pd_for_remote_vod=true

Then re-start Nimble Streamer. You can read more about config file in this article.

Test VOD streaming

Now you can test VOD streaming on a desktop and on a mobile device using HTML5 player and JWPlayer. Check "Test streaming media via multiple devices" section of "How to stream VOD with Nimble Streamer" article.

That's it.

Now you've set up a media server that can stream MP4 video files stored on remote HTTP Storage via HLS and MPEG-DASH protocols.

Take a look at Nimble Streamer performance tuning basics to see examples of live streaming resources usage and also learn about basic approaches to best performance.

In addition to single unit efficiency, you may create robust delivery networks by using load balancing techniques for spreading the load between several Nimble instances.

You can use "vod_cache_timeout" and "vod_cache_min_storage_time" parameters available in Nimble Streamer settings to define maximum and minimum cache lifetime periods.

You can also take a look at the video where Jan Ozer of Streaming Learning Center and Yury Udovichenko of Softvelum discuss dynamic packetizing of live, VOD and DVR content. It gives more details about Nimble Streamer VOD streaming, especially about remove HTTP origin feature set, it's pros and cons.

SMIL files support

Nimble Streamer can process not only media files, but also SMIL files located in remote HTTP storage. This works for both HLS and MPEG-DASH. You can use this feature as a convenient way of providing your viewers with different bitrate options for video playback.

Server-side ads insertion for VOD

Nimble Advertizer allows dynamically insert ads into VOD streaming. It supports pre-roll, mid-rolls and custom business logic based on programmable handlers. VOD files from remote HTTP location are also supported for SSAI.

Troubleshooting

Nimble Streamer internal transmuxer buffer sometimes may not be enough to fit generated HLS chunk. This may lead to this error in Nimble Streamer logs:

[tmux1] E: buffer is too small to generate

By default the buffer is 40MB. You can increase it using "vod_transmuxing_buffer". Read Nimble Streamer configuration description for config setup details.

Related documentation

Nimble Streamer, Video-on-Demand in Nimble Streamer, MP4 transmuxing to HLS VOD streaming, Create ABR VOD HLS with SMIL, HLS re-streaming set up, HLS, DASH and Icecast streaming load balancing, MPEG-DASH streaming via Nimble

Protecting media links from web scraping

Security and billing are often considered as most valuable tasks for every media business. It's hard to imagine a company that doesn't care about protecting its own content, which in fact underlies company's gain. Unprotected content can be used by other people for commercial purposes and that actually means missed profit for the content owner.

WMSPanel team provides their customers with set of paywall features for Nimble Streamer and Wowza, those are dedicated to solve the above tasks. The best-known feature is hot-linking protection, which allows to protect specified links from re-streaming and further re-use. Briefly, media URL is accompanied with special signature, which contains information about IP address of viewer and period of time the URL is valid for. So, protected URL can't be used to retrieve media content from any different IP address and expires after predefined period of time. Perfect!

However sometimes, our customers report, that hot-linking protection is broken, because they've seen an application, playing their protected streams. The application's name is usually Kodi, SimpleTV or PlaylisTV, but that isn't a full list. Reported case looks like re-streaming, but in fact it's not. Those applications don't produce media URL, but scrape it from a web page, which is in most cases accessible for everyone. Roughly speaking, they act as "man-in-the-middle", requesting web page from viewer's IP address, taking URL from that page and pasting it inside its own media player.

So, nothing to worry about? Not exactly.