Handling fuzzy FPS to get proper bitrate output

Nimble Live Transcoder allows creating transcoding scenarios with various transformations, including control over output bitrate. Usually you have some input stream with expected parameters like frame rate (FPS) and resolution and want to create one or more output with certain bitrate.

Normally you can define output bitrate using various parameters of encoder libraries, like it's described for x264 bitrate, constant x264 bitrate, NVENC bitrate and QuickSync bitrate setup in respective articles.

However, some sources may produce streams with uncertain frame rate. For example, mobile encoders may produce variable bitrate so when your users are streaming with apps like Larix Broadcaster you may get streams with unexpected FPS. As a result, if you ingest such fuzzy stream into a transcoding scenario, the encoder may produce the unexpected bitrate - higher or lower than you've defined in encoder settings.

You can use two approaches to handle this: by adding FPS filter or by defining FPS numerator and denominator parameters for encoder.

Option A: Add FPS filter

The best way to handle this is to set some definite value for FPS using a custom "fps" filter.
If you expect your users to stream with framerate around 25-30 FPS, you can set filter to 30 FPS. If you need a better framerate, just set it to proper value like 60 FPS. This filter uses optimized algorithm to insert extra duplicate frames, and encoders are optimized to process such repeating frames, so this will not add any processing overhead.

A.1 Add video_filter_reset_by_pts_enabled parameter to config

Before setting a filter, add video_filter_reset_by_pts_enabled = true parameter into your nimble.conf. With FPS filter, if you have MPEG-TS input (e.g. if you use SRT) and if input gets time gap, the output may hang up, so this parameter avoids that.

A.2 Add filter

Here's how you add the filter into transcoding scenario. Open your scenario and drag a Custom filter onto a pipeline. 

Then connect it to other filters and the encoder.

Now if you have your bitrate set up in the encoder element, you'll get the output bitrate which you've defined there after you save the scenario.

Option B: Using FPS numerator and denominator

Another option is to set output FPS numerator and denominator. All Transcoder encoders' libraries support them, so just set them as part of parameters: fps_n for numerator and fps_d for denominator. These parameters will "introduce" the encoder to the numbers which it will use for further bitrate calculation.

If you need 30 FPS, use fps_n=30 and fps_d=1, as shown below. The same formula applies to 60 FPS: use fps_n=60 and fps_d=1.

In this example, once you save encoder setting and then save scenario, the encoder will be likely to produce 1Mbps output based on expected frame rate of 30 FPS.



Let us know if you have any questions about these approaches.

Related documentation

Nimble Live Transcoder, Live Transcoder documentation reference,

RIST protocol in Nimble Streamer

RIST (Reliable Internet Streaming Transport) protocol is a new protocol for low-latency live video over un-managed networks. You can find out more about this protocol in RIST website. RIST is developed and promoted by RIST Forum where Softvelum is an affiliate member.

RIST is available via libRIST open-source library so Softvelum uses it to integrate this technology into Nimble Streamer media server. We give our customers all delivery options available on the market so libRIST is now one more delivery protocol in our stack.

Nimble Streamer allows both receiving (Listen and Pull) and sending (Listen and Push) RIST streams.

All supported protocols can be used as input for re-packaging into RIST: SRT, RTMP, RTSP, MPEGTS, HLS and Icecast.

Having any RIST input stream, it can be re-packaged into all supported output protocols: HLS, MPEG-DASH, SRT, RTMP, RTSP, MPEGTS and Icecast, as well as recorded into DVR.

This article describes the steps to set up RIST streaming via Nimble Streamer.

Install Nimble Streamer and RIST package

RIST is available via separate package for Nimble Streamer. You need to install it in order to use this protocol.

1. Sign up for WMSPanel account if you haven't done it yet.
2. Install Nimble Streamer or upgrade it to latest version.
3. Follow instruction RIST page to install RIST package. RIST package is currently available only on Ubuntu 18.04 and later versions.

Now you may proceed with the setup.

Receiving RIST streams

Being logged into WMSPanel, click on Nimble Streamer -> Live streams setting top menu. Then choose MPEGTS In tab.

Click on Add RIST stream button to open the dialog shown below.

RIST provides two modes for obtaining the stream:

• Listen sets Nimble to wait for incoming data and process it as soon as it arrives. Your source needs to be set "Push" mode.
• Pull sets Nimble to initiate the source server to start sending the data. Your source needs to be set to Listen mode.

If you select Listen, you'll need to specify the local interface - Local IP and Local port - which Nimble will listen to in order to get a stream. If you'd like it to listen to all interfaces, just set Local IP to 0.0.0.0.

Notice: if you use RIST Simple profile then you need to use even-numbered port.

Alias is the name that will be used in incoming streams list.

Also check Add outgoing stream checkbox and fill in the Application name and Stream name fields if you'd like to automatically create outgoing stream for further processing. This step is specific to MPEG-TS streams in Nimble.

If you'd like get stream in Pull mode, you'll need to specify the Remote IP and Remote port where you'll get your stream from.

Notice: if you use RIST Simple profile then you need to use even-numbered port.

Once you save setting, the settings will be applied  create outgoing stream, you can use a breadcrumbs on top of settings page to go to live streams of specific server (by clicking "RIST demo" server link in this example):

You can now use this stream for further processing and playback similar to other MPEG-TS processed streams.

Sending RIST streams

We assume that you have a live stream ready to be sent out. If you don't, please refer to live streaming scenarios to see how you can set it up.

Go to UDP streaming tab to add new setting.

Click on Add RIST setting to see dialog below.
RIST allows streaming in two modes:

• Push will initiate the active sending of the selected channel to the destination IP/Port which is working in Listen mode.
• Listen will make Nimble Streamer wait for the Pull command from remote server.

Mode is a field to set this behavior.

If you choose Push mode, you'll see Remote IP and Remote port fields to specify the destination.

If you choose Listen, you'll be able to specify Local IP and Local port to be used for listening for a connection. Just like in receiving use case, If you'd like it to listen to all interfaces, just set Local IP to 0.0.0.0.

Notice: if you select RIST Simple profile then you need to use even-numbered port.

Source application name, Source stream name and their respective PMT PIDs, Video PIDs and Audio PIDs describe where the content is taken from for further transmission. Those streams need to be defined prior to making this setup. You may define multiple sources - in this case RIST channel will have multiple streams in it.

Supported RIST parameters

Nimble Streamer allows using the following parameters. You need to set them both on sender and receiver side. You can refer to RIST documentation for more details and tips on their usage.

• aes-type is encryption type, set if only secret is not empty. Available values are are: 0 - none, 128 - AES-128, 192 - AES-192, 256 - AES-256.
• bandwidth is RIST recovery bandwidth (Kbit/s), it's 100000 by default
• buffer is maximum RIST recovery buffer size in milliseconds, it's 1000 by default.
• cname is a manually configured identifier.
• compression defines compression, 0 is for disabled, 1 is for enabled.
• keepalive-interval is a Keep-alive timeout in milliseconds, it's "100" by default.
• miface is multicast interface name.
• profile is RIST profile. Available values are are: simple, main (default). Notice that if you use simple profile then you need to use even-numbered port.
• reorder-buffer is reorder buffer size in milliseconds, it's 25 by default.
• return-bandwidth is RIST recovery return bandwidth (Kbit/s), it's 0 by default
• rist-logging is logging level. Available values are are: quiet (default), info, error, warn, debug, simulate
• rtt is round-trip-time (RTT) in milliseconds, it's 500 by default.
• secret is encryption password.
• session-timeout is a session timeout, in milliseconds.
• rist-stats-interval sets interval in seconds for reporting stats used by Nimble native API RIST methods for obtaining current sender and receiver stats for RIST streams. In order to use these API calls, you need to add parameter into RIST output and input respectively.

If you have any questions about these parameters, please refer to librist documentation as describe below.

RIST sender and receiver statuss

You can use Nimble HTTP API to obtain RIST sender and receiver statuses.

libRIST

Nimble Streamer uses librist library version 0.2.0. If you have any questions on RIST parameters usage and other topics, please refer to RIST documentation. and contact RIST team about it.

Related documentation

Softvelum Nimble Streamer, MPEG-TS support in Nimble Streamer

Nimble Streamer DRM with Widevine, FairPlay, Playready, EZDRM and VCAS

Softvelum team is glad to announce that Nimble Streamer now has full support for a number of Digital Rights Management (DRM) capabilities for live streaming.

The following encryption engines are supported to protect MPEG-DASH:

• Google Widevine™
• Microsoft Playready™

Also, Apple FairPlay is supported to protect Apple HLS streams.

All encryption is supported for H.264/AVC and H.265/HEVC codecs.

The following key management servers are supported:

• Widevine Cloud Service with key rotation
• EZDRM™ key management support for Widevine, FairPlay and Playready

Also, previously supported Verimatrix™ VCAS key management for HLS protection is fully supported.

DRM configuration is performed via easy-to-use configuration file which allows defining per-application setting for all available DRM features.

Visit DRM feature page to learn more about DRM setting.

Watch this video tutorial showing EZDRM setup for live, DVR and VOD content protection.

Nimble DRM is part of Nimble Addenda premium package which requires a license accessible via monthly subscription. Addenda covers Nimble DRM and Nimble Advertizer capabilities.

Please read Nimble DRM spec and subscribe for a license to try it in action.

Related documentation

Nimble Streamer DRM, Nimble Addenda, Nimble Advertizer, 

iCloud support and file operations in Larix Broadcaster for iOS

Larix Broadcaster for iOS now has improved capabilities for recording live stream into local files.

Here are the options you can use for files storage:

• Local storage available via MacOS Finder as described in this article.
• iCloud Drive
• Photo Library

You can also split video into sections, like it's usually done in dash cams. The recording is divided into multiple files by length.

Here's how you can set this up.

First, install Larix Broadcaster from AppStore.

Go to app Settings / Capture and recording / Record menu.

First, turn on the Record stream to automatically record any live stream.

Tap on Storage to select the default storage for recorded videos and screenshots. This will be one of 3 options mentioned above.

Split video into sections allows defining the length of recording which the stream will be split into. By default of Off.

Once you make recordings, go to Settings / Manage saved files menu.

Here you can long tap on file name to move it to proper destination. You can also tap Edit to perform multiple files operation. The iCloud tab will show the content of respective iCloud Drive folder. The recorded or copied files an be found in respective folders.

Take a look at a brief video overview of this feature.

Let us know if you'd like any improve in this feature set.

Related documentation

Larix Broadcaster, Larix documentation reference, Softvelum YouTube channel,

SRT FEC (forward error correction) support in Nimble Streamer

Softvelum is an active adopter of SRT technology and Nimble Streamer has extended support for it.

One of the features introduced in latest SRT library versions is the ability to set custom packet filters for SRT transmission. The first introduced built-in filter is Forward Error Correction (FEC).

Before using feature, please read carefully the SRT Packet Filtering & FEC feature documentation in SRT library github repo.

1. Disclaimer

We assume you are already familiar with SRT setup and usage, and you've successfully used SRT in other scenarios and use cases.

Before proceeding further, set up a test streaming scenario and make sure it works without any filters.

FEC filter is still under development and we've added it per requests from our customers.
Here is what you need to consider before using it:

• Use FEC filter feature on your own risk.
• It may crash the server, so if you face any issues, check Nimble Streamer logs to analyse the problem.
• Read Known issues section below in case of issues.
• Try using it with test servers and test streams first, and then move to production only when you make sure it works as expected.

2. Upgrade

In order to use this filter, you must upgrade Nimble Streamer and make sure you have the latest SRT library package.

1. Nimble Streamer version must be at least 3.6.5-6, use this procedure to upgrade.
2. SRT library package must be at least 1.4.1, use this page to get upgrade instruction.

Once you upgrade and re-start Nimble Streamer instance, you may proceed to further setup.

3. Setup details

According to information from SRT developers team, the FEC filter must be set on both sender and receiver sides and at least one side should define a configuration for it. In our example we'll define configuration parameter on sender.

As was mentioned, we assume you've set up your SRT streaming scenario. Let's modify it to set up sender part.

3.1 Sender

Go to "UDP streaming" tab on "Live streams setting" page and open your SRT setting. Scroll down to parameters list and add new "filter" parameter with a value which you find appropriate, as shown on a screenshot below.

We use "fec,cols:10,rows:5" there just as example, but you can use any other value which you find appropriate for your case, please refer to FEC documentation to learn more.

As you see we use latency and maxbw parameters as described in this article, we highly recommend always using them for your cases as well.

3.2 Receiver

Now on a receiver side, you need to define "filter" parameter with "fec" value. Notice that you don't need to set more details because you've defined them earlier on sender side.

In case of Nimble Streamer setup, go to "Live streams settings" page, "MPEGTS In" tab and add incoming stream. Then enter "filter" parameter with "fec" value as shown below.

This was an example of FEC usage in Nimble Streamer.

4. Known issues

In case of any issues please analyse Nimble Streamer logs to get more details for further analysis.

As this feature is under development, it has a number of issues. We've faced one of the issues during testing.

When FEC is enabled and lots of packets are dropped, it does not recover:

If a large drop is simulated on the line, SRT gets it's self into a state where it's no longer transmitting packets:
The following message is displayed:
16:40:22.279871!W:SRT.c: FEC: LARGE DROP detected! Resetting all groups. Base: %1583134490 -> %1583139490(shift by 5000).

It's still not fixed as of April 3rd, 2020:

So if you find this error in your Nimble Streamer logs, just disable FEC filter for now.

This is also the advise for any other issues related to FEC: if you face any uncertainly, just remove FEC filter and use SRT without it because we won't be able to help you.

You can look for existing issues and solutions in SRT issues on github. You can post your questions there in case of concerns with FEC filter.

Related documentation

SRT support in Softvelum products, SRT in Nimble Streamer,