October 4, 2017

Troubleshooting Live Transcoder

Live Transcoder for Nimble Streamer is being used by various companies around the globe in many live streaming scenarios. However, some customers run into typical problems which they address to our support.

Some typical troubleshooting techniques for Nimble Streamer itself are described in Before you post a question to helpdesk article.

We'd like to describe Live Transcoder issues that appear in some cases and can be solved by the customer.

Q1: Stream stops unexpectedly

Nimble Streamer goes offline, it's shown as "offline" in WMSPanel account.

Check Nimble Streamer logs located at /var/log/nimble/nimble.log

Look for this type of messages:
[2017-10-04 04:23:26 P1486-T2435] [encoder] E: encoder too slow, droped 2038/2039 audio frames for [live/stream]
[2017-10-04 04:24:58 P1486-T2435] [encoder] E: encoder too slow, droped 3765/3766 audio frames for [live/stream]
You can also check system logs in /var/log/kern.log or /var/log/messages (depending on your Linux distro) to find this type of messages
Oct 04 04:25:28 loft24330 kernel: [46756.231449] Out of memory: Kill process 1486 (nimble) score 957 or sacrifice child
Oct 04 04:25:28 loft24330 kernel: [46756.249823] Killed process 1486 (nimble) total-vm:52904636kB, anon-rss:32208832kB, file-rss:0kB
Both errors indicate that your server cannot encode all streams in time. The second system error shows that Linux Kernel shut down the process due to Out of Memory condition.

As a resolution, you should decrease the number of encoded streams on this server. You could set up another server and encoder and make those extra streams be processed there.

Speaking of the memory, you should also check and set up proper RAM cache size to and make sure that value is enough to store your live stream chunk cache.

Q2: Unsupported format 12

You may see errors like these
[2017-10-13 22:18:14 P86076-T96639] [encoder] E: failed to encode video frame for [live/stream]
[2017-10-13 22:18:16 P86076-T96641] [encoder] E: unsupported frame format, 12 
This means you need to change pixel format.
Open your scenario and add new custom parameter "format" element prior to your encoder element. The parameter name will be format and the value will be pix_fmts=yuv420p as shown on this picture.

Once you apply settings, the error error should be gone.

Q3: NVENC is not available

If you fail using NVENC as you decoder and/or encoder, you may see messages like this:
nvenc is not available
This means you don't have NVENC driver installed or properly set up in your system.

Make sure you have the right packages installed. Those would be nvidia-encode-<driver_version> and nvidia-decode-<driver_version>, e.g. nvidia-encode-525.

Q4: What NVENC drivers should be used?

We highly recommend using official NVidia drivers.
We may decline requests for technical support in case if a customer uses any un-official or "patched" drivers from any third-parties.

Q5: Transcoder not found after installation on Windows

Some customers install Live Transcoder on Windows 2008 and after registering the license they still get a message like
Please install 'nimble-transcoder' package
with additional info about installation procedure.

First please try re-starting Nimble Streamer and try that again.
If that doesn't help, you may need to install update KB2533623 by Microsoft. Please follow this instruction to make update and then re-start Nimble Streamer.

Q6: Failed to create NVENC context

If you use NVENC and experience some issues you may see messages like these:
[2017-10-18 12:32:54 P23976-T24004] [tranmain] E: failed to create cuda context, gpu=1, flag=4
[2017-10-18 12:32:54 P23976-T24004] [tranmain] I: create cuda ctx, gpu=1, flag=4...
[2017-10-18 12:32:54 P23976-T24004] [tranmain] E: failed to create nvenc ctx for gpu=1, flag=4, res=2
[2017-10-18 12:32:54 P23976-T24004] [tranmain] E: failed to create cuda context, gpu=1, flag=4 
The res=2 flag means that NVENC ran out of memory and couldn't create proper amount of contexts for content processing.

You need to use context sharing approach described in this article.

Q7: Failed to encode, status 10

You may see these messages when using NVENC:
[2017-10-16 10:58:36 P138405-T141721] [encoder] E: failed to init encoder=0x7f1b540008c0, status=10
[2017-10-16 10:58:36 P138405-T141721] [encoder] E: failed to encode video frame for [live/stream_1080]
[2017-10-16 10:58:36 P138405-T141721] [encoder] E: failed to flush nvenc encoder
This also indicates too many contexts in use. Please refer to Q5 for more details on how to use context cache and find the right number of contexts.

Q8: Video and audio are un-synced

Sometimes streams are published to external destinations like Akamai or other CDNs with un-synced video and audio. This may be caused by the originating encoders.

Try applying interleaving compensation to your streams to eliminate the un-sync.

Q9: Closed captions are doubled up after FPS change

When a stream is up-scaled via filter (e.g. from 29.97FPS to 59.94FPS), the closed captions are doubled in the video stream. Closed captions are embedded into frames, FPS filter duplicates frames with all side data, and this leads to duplicated characters.
To fix that, add video_filter_preserve_cc_enabled = true parameter into Nimble configuration file, read this reference page for more details about Nimble configuration.

Feel free to contact us if you have any other questions or issues with Live Transcoder.

Related documentation

Live Transcoder for Nimble Streamer, Nimble Streamer performance tuningLive Streaming featuresZabbix monitoring of Nimble Streamer with NVidia GPU status.

No comments:

Post a Comment

If you face any specific issue or want to ask some question to our team,

This will give much faster and precise response.
Thank you.

Note: Only a member of this blog may post a comment.