January 25, 2017

Binding un-synced video and audio sources in Live Transcoder

Live Transcoder for Nimble Streamer has wide range of content transformation features which can be used in transcoding scenarios.

Some scenarios may require to take unrelated and un-synchronized sources of content and bind them together into a synchronized stream. This capability covers some major use cases like the following to name just a few:

  • YouTube does't accept audio-only and video-only content, so you need to add missing video/audio in order to comply.
  • Take video stream (e.g. a game footage) and put commentator's voice on top.
  • Online radio graphics overlay: take Icecast online radio stream, put still picture as video and publish it on a website or an external CDN so common video player could play sound and has was visual representation.
  • Take surveillance camera video stream, insert some sound of silence from MP3 or MP4 file and publish it to external destination - like aforementioned YouTube or CDN.

Nimble Streamer Transcoder is currently capable of both transforming file content for further live streaming usage and synchronizing it. Let's see how you can do this.

Installing Nimble Streamer Live Transcoder

First of all you need to have Nimble Streamer installed, as well as Live Transcoder

You can check basic principles of Transcoder setup using our video tutorials in this playlist.

Engaging graphics and on-demand content into live transcoding

Nimble Streamer can transform the following file types into live stream:

  • GIF
  • PNG
  • JPEG
  • BMP
  • TIFF
  • MP4 container with H.264 video and AAC or MP3 audio
  • MP3 audio files for audio streams

Video decoder

To use files in transcoding scenario you need to add the video decoder element to your scenario and choose File as decoding source as shown below.

Adding picture as a source for video decoder
You will see File path input field for entering full local path to a source file.

For still images (GIF, PNG, TIFF, JPEG) the FPS field specifies the key frame rate which will be used for video stream. In case you have GIF animation and its frame rate differs from the one you specify, then its playback speed will be higher or lower. E.g. if you have 15 images per second in your GIF and set FPS field to 30 then your GIF will be animated twice faster. The Stream ID parameter is ignored for still images.

For MP4, the FPS parameter will be ignored while Stream ID can be used in case you have multiple video tracks. In that case you will just set the value to track number, starting from 0.

The Decoder field is used for defining the engine used for decoding the input - currently it's either Default software decoder or NVidia. Read more about software decoder threads and NVidia decoder settings. The Threads field is used for software default decoder.

Audio decoder

Audio source is set similar way. drop Audio decoder element to the scenarios workspace and choose File.

Adding audio file as a source for audio decoder

You can specify File path to audio content. If you use MP4 with multiple audio tracks as audio source, you can specify the Stream ID to set it.

Synchronizing sources to audio

Having any source of video which is not in sync with audio, you can bind them together. This might be streams from file content which we described earlier. Also that might be original live streams from different sources which you need to be in sync in order to be played by all major players.

There are 2 major approaches for syncing streams - you can either sync audio to video or sync video to audio. The difference is about which stream will be used a primary source for timing synchronization.

Video to audio

This is the preferred option when you have audio as your main source of content and you need some auxiliary video to be shown. A good example is online radio streamed via Icecast which you want to publish to some delivery service which requires both audio and video in their streams, e.g. CDN or YouTube.

You need to create scenario as shown below.

Binding picture to audio stream

As you see you have audio passthrough for audio and then a decoder input created per "Engaging graphics..." section above. Video pipeline also has scaling filter to make a picture match designated size.
The encoder set up is shown below, check out Sync related streams field. It appears if you click on Expert setup link.

Encoder settings for video made from a picture

Sync related streams field needs to be set to Video to audio value. If you save this encoder and then open audio encoder settings, you will see that this field has also been set up to the same value to avoid sync-up collisions.

You can see audio being passed through, however you can create any other audio pipeline, including filtering etc.

Watch this video to see how this scenario is set up step-by-step:

You can find it in our YouTube channel.

Audio to video

This sync-up can be used when you have a source of soundless video which needs to be accompanied by some audio track. Surveillance camera streaming a good example for this.

The setup is similar - check scenario below as example taken from Audio decoder section above.

Binding audio file to video stream

Here you see video passthrough as video content is not touched and then audio created from file and encoded separately. The encoder setup is shown below.

Encoder settings for audio made from a file

Here you see Sync related streams field set as Audio to video.

Like in previous scenario, you can see video being passed through, however you can create any other audio pipeline, including filtering etc.

Watch this video to see how this scenario is set up step-by-step:

You can find it in our YouTube channel.

"Equalize-only" scenario

Of course you can synchronize video and audio from original live streams. This can be used when you do a voice-over with comments etc. The setup looks like this - just make sure both output streams names are the same.

Passthrough-only scenario to bind audio and video

And either audio encoder or video encoder need to have Sync related streams to either or two mentioned values - Audio to video or vise versa.

Encoder settings to bind audio to video stream.

This is basically how you can easily set up streaming from file sources and make streams synchronization.

Check this video for step-by-step instruction:

You can find this and other tutorials on our YouTube channel.

Feel free to visit Live Transcoder webpage for other transcoding features description and contact us if you have any question.

Related documentation

No comments:

Post a Comment