September 15, 2020

Building Quick Sync-only pipeline with Nimble Transcoder

Live Transcoder for Nimble Streamer provides many features for transforming live content using both software libraries and hardware acceleration.

Intel® Quick Sync is fully supported in Live Transcoder for decoding and encoding but all filtering operations were performed using CPU. That caused extra resources usage to transfer processed frames among CPU, GPU and RAM.

Nimble Live Transcoder now allows building transcoding pipelines which are performed completely with Quick Sync hardware acceleration. This is done using specific FFmpeg libraries which we use in addition to our own code.

This article shows how to set up this Quick Sync-powered processing chain.

1. Installation and initial setup


We assume you've already set up Nimble Streamer, it's been set up to get an incoming live stream and you've tested basic streaming. In our example we'll use a stream whose application name is "input" and stream name is "source".

If you're not familiar with Live Transcoder, take a look at Transcoder documentation reference.

Notice that the described functionality is available on Ubuntu 20.04 only. We'll support other upcoming LTS Ubuntu releases as well.

The basic steps to make Quick Sync working are as follows:
  1. Create a transcoder license and subscribe for it.
  2. Install Live Transcoder add-on.
  3. Create some simple scenario with CPU transcoding (e.g. downscale your stream to 240p). This way you'll make sure the transcoder was set up properly.
Now create a new scenario to start a new pipeline setup.

2. Decoder setup


Once you create a new scenario, drag and drop a blue decoder element onto the dashboard. There you need to specify "quicksync-ffmpeg" in Decoder field.


That's it. Now let's set up filtering.

3. Using filters


Once the frame is decoded you can process it via a set of ffmpeg filters which will work via Quick Sync. Nimble Transcoder supports a number of those, here are the most frequently used. Notice that you can refer to FFmpeg source code for more details about custom filters. As of September 2020, Nimble Transcoder uses FFmpeg 4.3.1 for some of its operations, including Quick Sync filters.

"Split" - allows creating several identical outputs from input video. It's available as a filter element in a tool box of Transcoder UI.

"Picture" - another filter available via UI element. Its setup for Quick Sync will look like this:



"fps" filter sets the frames per second rate and is defined via a custom filter. Its name is "fps" and the value is "fps=<number>", like "fps=30".

"scale_qsv" filter allows resizing the image. Add a custom filter into your scenario, set name to "scale_qsv" and then use filter parameters of your choice separated by comma. Common parameters are "w" (width) and "h" (height).


Other options include "format", "mode", "low_power", "hq". Please refer to FFmpeg sources file vf_scale_qsv.c for more details.

"vpp_qsv" filter allows transforming the content in a lot of ways, some of its parameters include "deinterlace", "denoise", "framerate" as well as flipping, scaling and a lot more. Please refer to FFmpeg source file vf_vpp_qsv.c for a full list of options.

4. Encoder setup


In order to encode video using Quick Sync within the hardware pipeline, you need to define Encoder as "FFmpeg" and set Codec to "h264_qsv" for H.264/AVC codec.


You can then define its custom parameters like "profile" for encoding profile , "b" for bitrate and many others which you can find in FFmpeg source file qsvenc_h264.c.

If you need to encode to H.265/HEVC, your Codec field must be set to "hevc_qsv".


It also supports a number of parameters like "b" for bitrate and you can find a full list in qsvenc_hevc.c from FFmpeg sources.

Once you finish your scenario and save it, you'll be able to transform your incoming video stream using only Quick Sync hardware acceleration.

When you have a video pipeline set up, you need to define the audio part. If you don't need any sound transformation, you can add a passthrough for it just like it's described in other setup examples.




If you have any questions, issues or questions, please feel free to contact us.

Related documentation


Live Transcoder for Nimble StreamerTranscoder documentation reference,

September 10, 2020

Make thumbnails for live streams

Nimble Streamer now allows generating thumbnails on-the-fly for any outgoing stream. At the moment it's a single-frame MP4 file which can be embedded into any web page via <video> tag.

In order to start generating those thumbnail, you need to enable Generate MP4 thumbnails parameter for either Global setting of the server or for specific applications' settings as shown below.

Once the parameter is enabled, you can access the generated thumbnail via this kind of URL:

http://server_URL:8081/live/stream/thumbnail.mp4


Here are the examples of global and per-application "Generate MP4 thumbnails" setting enabled.  The thumbnails are re-newed within a Interval period which you set up after checking the checkbox, by default it's 6 seconds.


Here's application setting:



Related documentations

Live streaming in Nimble Streamer