libaom-av1 and SVT-AV1 transcoding setup with Nimble Streamer

AV1 format is a promising solution for high-quality efficient video compression.

Now with Nimble Streamer support for AV1, streaming services can easily adopt this advanced format to deliver high-quality content. Adding another significant piece into AV1 support, here we describe the transcoding options available for Nimble Streamer Live Transcoder: SVT-AV1 and libaom-av1 encoders are now supported via the respective FFmpeg encoder option.

Prerequisites

Before using the Transcoder for Nimble Streamer, you must have a basic subscription to WMSPanel and a Transcoder license. Please find more details on installing and registering an instance of Transcoder on the following installation page. You should also have a basic knowledge of Live Transcoder setup, please refer to our documentation reference.

1. Decoding AV1 received streams

AV1 streams are supported as video input, and no extra setup steps are required. Just drag Video Source to a timeline, and specify the name of a stream for decoding. After that any of the Transcoder’s filters can be used on the stream.

2. Encoding with libaom-av1

Specifying libaom-av1 for encoding is not much different from any other encoders. Add the Video output to a timeline, name the output application, and stream, and then select FFmpeg as the encoder. Next, type in libaom-av1 in a Codec field. Once you click OK and then Save the scenario, the settings will be applied and you will get the output stream in AV1 format.

Here are the parameters you can use for it

2.1 libaom-av1 constant quality 

To set constant quality for the encoding process, fill in parameters as ‘crf’ and then the number in the option field.

Accepted values are 0-63. Lower values = better quality, 0 = lossless. CRF 23 ≈ x264 CRF 19 (visually lossless).

This mode ensures that each frame receives the appropriate number of bits to maintain a consistent quality level. It is set the same way as for x264 or x265.

2.2 libaom-av1 constrained quality

This one maintains desired video quality (set with ‘crf’ option) while targeting a bitrate limit (‘b’). Use for consistent encoding with a bitrate cap.

The ‘b’ value must not be 0.

Alternatively, you may specify the minimum and maximum bitrate range instead of the ‘crf’ option:

Notice that this mode doesn’t mean setting strict limits for output bitrate, and it can be even higher than  the input depending on the source’s data.

That’s a frequent question to our support team along with the question if the bitrate is specified via the ‘b’ param only (even if no ‘crf’ or ‘minrate’/’maxrate’ is set) and expected to be strictly constant.

2.3 libaom-av1 keyframe control

By default, the Transcoder suggests encoders to use keyframes as set by the ‘Keyframe alignment’ option. 

You may force keyframes via the encoder parameters also. The ‘g’ parameter can be used to set the maximum keyframe interval.

To get a fixed Interval specify both ‘g’ and ‘keyint_min’ to the same value. 

To force all frames as keyframes use ‘g’ as  ‘0’.

You can find more information about libaom-av1 on this page.

3. Encoding with SVT-AV1

SVT-AV1, also known as libsvtav1, is a video encoder that Intel and Netflix created together. In 2020, the Alliance for Open Media (AOMedia) chose it to lead the way for AV1's development and future video codecs. It's also supported by Nimble Streamer Transcoder.

To specify SVT-AV1 encoding, add Video Ouput, name the application, and stream for the output, choose FFmpeg as encoder and type ‘libsvtav1’ in the Codec field.

As you click ‘Ok’ and then ‘Save’ in the scenario window, you will receive AV1 output produced by libsvtav1 codec.

Here are some additional parameters to consider.

3.1 SVT-AV1 CRF

The 'crf' parameter ensures each frame receives the appropriate number of bits to maintain a consistent quality level.  It is set the same way as for x264 or x265, or as described above for libaom-av1.

Accepted values are 0-63. The default value is 35. Choosing a lower CRF value results in higher video quality and a higher bitrate. Encoding in a completely lossless mode is not supported at the moment.

3.2 SVT-AV1 Presets

In SVT-AV1, the ‘preset’ option is used to balance encoding speed with compression efficiency. There are 14 presets available, numbered 0 to 13, where higher numbers mean faster encoding. 

3.3 Additional options with SvtAv1Params

There are some additional options you may want to try with the SVT-AV1 encoder for getting either faster encoding or better quality.

They are listed in the section ‘Usage of SvtAv1Params’ on the following documentation page.

Here we describe, how to apply it in Nimble Transcoder.

Type in ‘svtav1-params’ as a parameter of Video Output in the encoder settings. Set the value of the desired SVT-AV1 parameter and its value in an options input. You may chain several options via semicolons like ‘option1=value1:option2=value2…’

In the screenshot, we showed how to enable the ‘fast-decode’ option and the ‘tune’ option set to Visual Quality (0). The option string is ‘fast-decode=1:tune=0’

You may find additional info about SVT-AV1 on this page.

In conclusion, the arrival of AV1 encoding offers new capabilities with its superior compression, royalty-free nature, and high-resolution support. While processing power for encoding might be higher, the long-term benefits – from reduced bandwidth costs to exceptional video quality – make AV1 a future-proof investment for the live-streaming video landscape. Don’t hesitate to evaluate it right now with Nimble Streamer Transcoder.

Live video overlays and videowall with Nimble Live Transcoder

In the dynamic landscape of media streaming, enhancing content through creative overlays has become an indispensable aspect of media delivery. Nimble Streamer Live Transcoder emerges as a powerful tool in this realm, offering a versatile platform for superimposing text, images, animations, or live video streams while preserving and enhancing your brand identity. Here, we will talk about overlaying video or pictures on top of the video.

Prerequisites

Before using the Transcoder for Nimble Streamer, you must have a basic subscription to WMSPanel, as Transcoder does not work for trial servers. The Transcoder license costs 50 USD per server per month, there is no trial version for it.

Please find more details on installing and registering an instance of Transcoder on the following installation page.

You should also have a basic knowledge of Live Transcoder setup, please refer to our documentation reference.

Basics of operation

First, a base layer that determines the output resolution to create an overlaid video must be defined. This can be a live video or an image.

Then, all other input sources will be stacked on top of this main layer via the Overlay filter. You may also need the Scale filter to adjust the size of the other sources to place and fit them on the base layer.

The next concept is that you must define one of the sources as the Main stream. Once the scenario is started, the Transcoder will start equalizing all other streams’ timestamps to the Main stream to provide a stable source for Overlays.

Let's go through the details of this scenario setup below.

Sources for overlays: live streams

All of the supported source types can be used as a source for Overlay or backing layer: live streams, image or video files, and NDI sources.

Put a Video source decoder block on a timeline, and the Add video source dialogue will appear.

This will allow choosing the source type: Stream, File, NDI. First, we describe the regular live stream setup.

Type in an application and choose the stream to auto-fill the stream’s name. If the streams are registered in the panel, they will appear for selection. Choose the required one.

Next, you must also choose the purpose of this source: it could be either the Main stream or the PTS Adjusted stream.

Checking the Main stream box will mark the stream as main. For your convenience the main input will be marked with a bull's eye:

To equalize any other stream's timestamps with the Main's timestamps, choose the PTS adjustment enabled checkbox in its source’s settings.

Since the Main and PTS Adjustment are defined, you can safely chain several Overlays (or any other filters) in a Transcoder’s scenario. This versatility will help create distinctive visual effects for your video output.

As for the Audio, the Main is the only stream that will have perfectly synced audio. We do not advise taking audio from other streams, as we cannot guarantee the audio sync in such a case.

If the source live stream is over or stopped, the decoder will hold the previously available frame in its output, making the next in a chain Overlay filter operational.

In case the Transcoder scenario is just starting and one of the live sources participating in the Overlay chain is missing, the Scenario will fail to start. We advise using hot-swap failover to ensure the source always has some viable frames to process.

Source for overlay: video file, static, or animated image files

Move on to the next type of decoder: File Source decoder. Although we already have an article about decoding from files, here we will notice the crucial points.

As the name implies, it uses files on the same server Nimble Streamer is running as input. As you specify the path to some file and the scenario is running, then the File Decoder generates looped output with stable timestamps which can be assigned as Main. This is true for both video or image files. As you might have spotted, there’s no PTS Adjustment checkbox. It is considered that if the Main stream is not specified, such sources will be forced as PTS adjusted.

The video file formats supported are MP4, MOV and MKV. The codecs are listed in our codecs reference. Videos with alpha channel are not supported at the moment. Feel free to ask our support team if you plan using them.

Supported image formats include PNG, APNG, JPEG, GIF, TIFF and BMP. The alpha channel or GIF transparency is available. GIF and APNG animations are supported too.

In case of a single frame image, the file will be re-loaded as fast as the file is modified or overwritten. Notice that reloading will not work for video files or files with a sequence of frames, like APNG animations.

All the above makes the File Decoder a handy source for animated or updated Overlays like moving logos or bar ads.

Notice that path considerations are different between Windows and Linux.

For more information on File decoder, please read Binding un-synced video and audio sources article.

Single Overlay filter setup

Now let's move on to some practical setup that will be a basis for cases like video wall or mosaic.

We will combine a backing static layer with some live stream.

Suppose we require a standard 1080p output and on top of it a live video will be played with a resolution half that of a frame.

Prepare an empty 1080p resolution image, that will define a frame and put it in some folder accessible by Nimble. It could be a solid color, but for clarity, the color bars are used. We advise using PNG, as it is a more versatile format.

Put Video Source, select File, and type in path and filename. Set FPS to 30.

Add another Video Source and this time specify a received live stream by its application and stream names. We'll choose live/bunny here.

Next, apply a Scale filter to ensure that the live stream does not fill the background layer. Let’s make it 540p as it sounds applicable. Drag a line from the Live Stream Source decoder to the Scale filter.

Next goes the Overlay filter. Drop it on a timeline closer to File Source decoder. Connect the file decoder and Overlay filter by dragging a line between them. Specify the position of an overlaid video.

Set the output encoder by dropping Video Source to the timeline, name the output stream (like overlays/out) and choose any settings you like, default values are acceptable for checking.

One thing to note: Ensure that the Main video is received from: option refers to a File Source, as this specifies the source for the backing layer with a higher resolution.

Now it is time to decide which of the inputs to consider primary for timestamp adjustment.

The concern here is that the output stream could only have one audio source that would be synced with video and it must be marked as Main. So in this simple scenario, the preference naturally leans toward the live stream if we add its audio as an Audio source.

However, if syncing sound isn't necessary - let's say you just want background music - then specify the File source as the Main stream. In this case, the timestamps will be produced by the Transcoder itself and all other stream’s timestamps will be adjusted to them.

Click the Save button and in a few moments, you will have overlaid scaled video in the output stream via any Nimble-supported streaming protocol. Click Live Streams above the save button to jump from the Scenario to the List of Live Streams.

Multiple Overlay filter setup (videowall)

Now we're moving to a more advanced setup: chaining more video sources.

Add another Video Source (we use a file here for simplicity, but it's up to you), Scale, and Overlay. Use the same params as above, and chain them as shown in the picture. In the Overlay’s filter settings, make sure Main video is received from: is set as a previous Overlay filter.

You may add more video inputs to overlay them to form a video frame like below:

Hardware accelerated Overlay and other filters

Is the hardware acceleration available for this feature set? Absolutely, but just for Ubuntu Linux users. Both NVENC and QuickSync have hardware-accelerated versions of overlay filters. It's better to use NVENC-ffmpeg or NVENC-quicksinc decoders with these filters for performance reasons. This will avoid getting and putting frames to the system’s RAM and losing CPU cycles by using additional hwupload and hwdownload filters.

However, these decoders come with certain limitations. While they allow you to decode H.264 or HEVC (if supported by the GPU) mp4/mkv video files, they are not suitable for handling static images or animated files such as GIFs.

Remember to consider the hardware-accelerated scale_npp (scale_cuda) and vpp_qsv (scale_qsv) filters for resizing a video source. NVENC-ffmpeg decoder allows scaling in its settings by the resize option. Explore additional information about NVENC-only scenarios in the following article.

If you require more details on creating an NVENC-only scenario, please let us know. We have plans to publish an article specifically addressing this topic.

Troubleshooting

Here we’ll describe some tips and common errors you may face while while using Transcoder.

Although we did an extensive job of improving the stability of the Overlay filter, anyway we recommend using the out-of-process mode to provide better stability for the Nimble Streamer server. This mode will ensure that a single scenario with an issue won’t crash Nimble Streamer. Read about it in this article.

If you’re changing source type, let's say from File to Live, scenario restart is considered. It can be performed while editing the Scenario by the corresponding icons near its name.

Some common messages in the log:

• [video_decoder] E: failed to open stream /some/path/file.ext - You’ll get this error in the log if the specified file name is not found or Nimble has issues accessing it (e.g. due to permission restrictions).
• [video_filter] E: failed to parse filter graph - This means something is wrong with a filter chain from the parameter or there are filter name incompatibility errors between the decoder and filter. Other messages are needed for details.
• E: encoder too slow’  for encoder or decoder in most cases it means you are running out of processing power. If the multicore CPU is used, and you observe that some cores are not loaded when the error is present, this may be improved by allowing more cores for decoding or encoding.
• [video_filter] E: reset video filter on pts gap(prev=XXX, cur=YYY, timescale=1/1000), filter=XXXXX, stream=larix/2 - It's not an error, but the indication that due to the timestamp jumps, the filter was re-created.
• [video_decoder] E: decode interval gap detected, decode_interval=XXXX for [_some_path_] - it very likely is a decoding performance issue or the inconsistent system time.

What’s next?

You can place additional filters ahead of the Overlay filter to create visually appealing images that include lower thirds, L-shaped or bar images, watermarks, and even scrolling text. We'll cover this topic in upcoming articles.

Ogg/FLAC support in Nimble Streamer

Nimble Streamer Icecast streaming feature set covers many use cases and supports multiple codecs.

Now Nimble also supports FLAC (Free Lossless Audio Codec) with Ogg container for Icecast.

You may use it in various combinations:

• Take Icecast input with Ogg/FLAC content and then deliver FLAC Icecast content further to any number of listeners without changing the codec. All features related to content protection and monetization can be applied to the output Icecast.
• Decode FLAC via Nimble Live Transcoder, encode it into AAC or MP3 and deliver further via any supported protocol, including Icecast, audio-only HLS, SLDP or any other. You may also transcode it into Opus for WebRTC delivery. Notice that you'll need to upgrade your Transcoder package to do that.

This allows using Ogg/FLAC in any of the workflows needed to deliver high quality audio to your listeners.

Let us know of your experience with audio features of Nimble Streamer.

SRT multipoint listener in Larix Broadcaster

Softvelum products have wide support for SRT streaming, processing and playback.

Larix Broadcaster has various capabilities for SRT streaming including all modes - Caller, Listener and Rendezvous - and a number of SRT parameters.

SRT Listen mode now allows streaming as a multipoint listener on both Android and iOS.

It means that your consumers who want to get live content from same mobile can pull the stream simultaneously without waiting for connection to be available. This opens a use case when you set up your streaming mobile device and have various participants of the same production team to get same content with no need to re-set the connection settings.

If you choose to provide an SRT stream in Listen mode, Larix will also provide Max listener connections parameter to set control on how many consumers can access the stream simultaneously.

Try it now with Larix Broadcaster:

• Get in on Google Play
• Download on the AppStore

Let us know if you have any questions about this or any other feature of Larix products.

Axinom DRM in Nimble Streamer

Axinom DRM has recently been added to Nimble Streamer DRM bundle.

It allows protecting live and on-demand content with Widevine, Fairplay and PlayReady DRM encryptions.

Its setup is as easy as any other DRM, here's an example:

drm {
  application = live_drm
  type = axinom
  axinom_tenant_id = 0e51eeee-7921-5678-zzzz-xxxxxxxxxxx
  axinom_management_key = faa6dddd-1234-nnnn-yyyyyyyyyy
  content_id = 1232345-8602-11ea-b04b-7fccdabcdef1
  content_ids {
    stream1 = 1232345-8602-11ea-b04b-7fccdabcdef2
    stream2 = 1232345-8602-11ea-b04b-7fccdabcdef3
  }
}

You'll need to obtain Tentant ID and Managemetn key from Axinom.

Also notice that you need to specify the content IDs because it's being used for content licensing.

Let us know of your experience with Nimble Streamer DRM.