May 4, 2022

WebRTC WHIP publish setup for Nimble Streamer

In memory of Alex Gouaillard
who inspired our team for WebRTC

WebRTC has become a significant part of live streaming landscape in various use cases and scenarios - from low latency streaming to live chatting. It's a big stack of technologies which are combined in various combinations depending on the problem which a customer needs to resolve.

Softvelum team got multiple requests from customers regarding WebRTC support and finally we came to a combination of proper technology pieces best fit for solving this task. The streaming tasks which our customers specifically wanted us to solve are related to easy ingest of live streams from any browser.

Current WebRTC support in Nimble Streamer covers the following:

  • Ingest of WebRTC live stream into Nimble Streamer.
  • WHIP is used for signaling, see details below.
  • H.264, VP8 and VP9 video and Opus audio input.
  • H.265/HEVC video input from Apple devices.
  • AV1 video input from Chrome
  • JavaScript client for publishing video and demo page with sample client.

Signaling is an important part of WebRTC stack because it defines how a client connects to the host or to another client. Nimble Streamer uses WebRTC-HTTP ingestion protocol (WHIP) for signaling. It's a standard with Internet Draft status and it's already used by various WebRTC products. So we decided to use WHIP to be compatible with as many solutions as possible.

Nimble Streamer uses Pion implementation of WebRTC API. Special thanks to Sean DuBois and all Pion contributors.

Output streams can be generated in all protocols supported by Nimble Streamer, e.g. HLS, SLDP, NDI, SRT, depending on output codecs and required transcoding, see more details below.

Let's go step by step to set up Nimble Streamer to receive WebRTC ingest.


Notice that currently only Linux version of Nimble Streamer supports WebRTC. We're working on Windows support.

We assume you've already installed Nimble Streamer or upgraded it to the latest version. You also need and active WMSPanel account and a respective subscription.


1. Enable feature in Nimble config

First, you need to add a couple of parameters to nimble.conf to enable the feature. On Linux this file is available as /etc/nimble/nimble.conf. For more details about this file and its parameters, check Configuration reference page.

Add these parameters:

webrtc_whip_support = true
access_control_allow_headers = content-type
access_control_expose_headers = location
transcoder_change_params_on_the_fly_enabled = true

Then re-start Nimble Streamer instance. On Ubuntu it's done by this command:

sudo service nimble restart

Check installation instructions for other platforms.


2. Set up SSL for Nimble

The next step is to enable SSL for your Nimble instance as it's required for secure WHIP signaling.

You can set up your SSL certificate using this general instruction. You may obtain CertBot Let's Encrypt free certificate as we've described here.

For testing purposes you may create your own self-signed certificate but in order for it to work, you'll need first to open any https:// page like https://yourhost.com:port/ and accept security risk.

For production purposes in general, you need to have a valid SSL certificate. Also, your server must be assigned for the domain of this certificate.

Once you've set up SSL for Nimble, you need to test it. Open https://yourhost.com in your browser, where yourhost.com is the host of your Nimble. If you get error 404 and have no warnings from your browsers then your SSL was set up is properly and is working.


3. Set up WHIP client authorization

WHIP clients use URL parameters to pass their settings to the host.

WHIP client allows publishing from any browser to the server so Nimble Streamer requires to have user and password to be defined for the application where the publishing will be performed. If you don't set up a user/password credentials pair for the WHIP client and for the target application then your user won't be able to stream.

To set up an application, go to WMSPanel, open Live Streams Settings menu, choose a designated server, open Applications tab and create an application with required user and password.


In our example the app name is "live" and we'll use "whip" as stream name later on.

You may use two options to authorize clients on the server for publication.

3.1 Simple user/pass authorization

In your client publishing URL, use "whipauth" parameter to send credentials like this:

https://your_host/live/whip?whipauth=login:password

Where "live" is the name of the application with credentials.

Notice that whoever opens your publishing page, they can see your app name and user/password pair. This means high chance of leaking credentials for unauthorized publications. So use this authorization approach only if your debugging or if you provide separate applications for your trusted publishers.

In any other case, please use publish control framework.

3.2 Publish control framework

If you need more sophisticated authorization of your publishers based on your business logic, use Publish control framework. With publish control, you can prevent leaking of your publishing credentials. Also, you'll be able to get the status of all published streams and decline any of them any time.

When the setup is done, the URL will have "publishsign" parameter.

https://your_host/live/whip?publishsign=aWQ9SURfMSZzaWduPW95Zi9YVHBLM0c3QkQ4SmpwVnF1VHc9PSZpcD0xMjcuMC4wLjE=

Read this setup article to get all details.


4. Codecs support

As was mentioned earlier, Nimble Streamer supports H.264, VP8 and VP9 video with Opus audio in WebRTC ingest. So if your client uses these codecs, Nimble will be able to process them and make proper output.

In addition, you may use H.265/HEVC video input from Apple devices as well as AV1 browser publishing from Chrome.

If you need your users to publish from their browsers only with certain video codec, you can indicate that by setting "videocodecs" parameter. E.g. to make server accept only H.264 you can set it like this:

https://your_host/live/whip?whipauth=login:password&videocodecs=h264

If you are ready to accept either H.264 or VP8, use comma in that parameter's value:

https://your_host/live/whip?whipauth=login:password&videocodecs=h264,vp8


5. Generating output

Once the content is ingested, Nimble Streamer provides the following options for further processing.

5.1 Direct output via limited protocols

If the ingest has H.264 and Opus codecs, Nimble Streamer will be able to generate H.264/Opus output via these protocols:
  • MPEG2TS-based protocols: MPEG-TS over UDP multicast, SRT and RIST, to be played via VLC or ffmpeg.
  • SLDP: the protocol supports Opus, it can be played in SLDP web player on Windows, Linux and Android.

5.2 Full-featured transcoded output

All input codecs - VP8, VP9, H.264 and Opus - can be transcoded into any other codecs. That includes H.264/AAC output that is a de-facto standard for Internet, as well as generate HEVC (H.265) video. 

Use Nimble Live Transcoder to transform the input with any variety of decoders and encoders, with software libraries, NVENC and QuickSync hardware acceleration.

Watch Transcoder video tutorials for more setup examples. For those scenarios, the WebRTC ingest will be just another input stream. Here is a simple example of a transcoder scenario with H.264 and AAC output.






You can then use any combination of live streaming output protocols and options, like HLS, SRT, NDI, SLDP, etc. Nimble Streamer transmuxing engine will provide any combination you need.

You can also record the generated content using Nimble Streamer DVR and then provide the playback using HLS and MPEG-DASH protocols.

5.4 Notice on packet loss

Notice that if a publishing client and your server are located far from each other or need to communicate via bad quality networks, then you should expect some video and audio frames loss. Protocols and players handle this type of frames loss differently. At this moment Nimble Streamer does not try to add fake video frames or audio silence to compensate that behavior.

6. Network-related and general parameters

By default, Nimble Streamer works in ice-lite mode

If Nimble server instance runs on a host with public IP address then additional configuration is not needed.

If a server instance runs on Amazon EC2 then you'll need to create an additional config file at /etc/nimble/whip_input.json and add the following JSON there:

{
  "NAT1To1IPs":"a.b.c.d",
  "NAT1To1CandidateType":"host"
}

where "a.b.c.d" is a public address assigned to AWS server instance. If it has multiple IP addresses, just add them in the same parameters separating by comma like this:

{
  "NAT1To1IPs":"a.b.c.d,w.x.y.z",
  "NAT1To1CandidateType":"host"
}

This file is processed by Nimble at the beginning of each new publishing session, so you can change it without re-starting the server.

To define ports range, you can also add these parameters:

{

  "PortMin":1000,
  "PortMax":40000
}

In this case the candidates will be selected only from the range of ports 1000 to 40000.

If you want to use the same port for all WHIP ingest connections, you can use the following parameter instead of PortMin and PortMax.

{
  "ICEUDPMuxPort":1234
}

Notice that you cannot use the same port for both WHIP ingest and WHEP playback simultaneously.

If you use network parameters mentioned above, the combined JSON in this case will be:

{
  "NAT1To1IPs":"a.b.c.d",
  "NAT1To1CandidateType":"host",
  "PortMin":1000,
  "PortMax":40000
}

In addition you can use this config file to define supported codecs on the server level instead of defining them per session using SupportedVideoCodecs parameter:

{
  "NAT1To1IPs":"a.b.c.d",
  "NAT1To1CandidateType":"host",
  "PortMin":1000,
  "PortMax":40000, 
  "SupportedVideoCodecs":"vp8,h264" 
}

JSON format requires this kind of syntax and if you add them in different blocks or have no commas between parameters, Nimble will not process the config.


7. Browser publishing library and demo page

We've created a JavaScript library which you can use for adding publishing capabilities into your web pages. Use its code in your projects or take it as is for embedding into your pages to connect your users to Nimble Streamer.

There's also WebRTC publication demo page which uses that library to provide simple way to check your server setup. Just enter a WHIP URL with server address and publishing credentials, and then click on Publish. You will then be able to use your camera and microphone to streaming, and will see detailed logs of what's happening.

8. Mobile publishing client


Larix Broadcaster for iOS and Android allows ingesting WebRTC into Nimble Streamer or any other server/service with WHIP support with H.264/AVC video and Opus audio.

Watch Ingesting WebRTC from Larix Broadcaster into Nimble Streamer tutorial as an example of setup. There we make passthrough scenario with Opus audio ingested from iPhone and played in browser.

9. Video tutorials


Watch this brief tutorial demonstrating the setup process.



Also watch setup process to take WebRTC ingest and produce NDI output from it.


The following tutorial shows how to set up a Nimble Streamer to receive content via WebRTC and then send it as the UDP multicast into the local network without transcoding.



10. WHEP ultra-low latency playback

If you'd like to use WebRTC ultra-low latency for further playback, Nimble Streamer fully supports it.

Please read WHEP WebRTC low latency playback in Nimble Streamer article for more details regarding setup and usage.

WebRTC adaptive bitrate for WHEP is also supported in Nimble Streamer, read the article for setup details.

Our team keep improving WebRTC support in Nimble Streamer, so stay tuned for updates.

Follow us in social media to get updates about our new features and products: YouTubeTwitterFacebookLinkedInRedditTelegram

Related documentation


No comments:

Post a Comment

If you face any specific issue or want to ask some question to our team,
PLEASE USE OUR HELPDESK

This will give much faster and precise response.
Thank you.

Note: Only a member of this blog may post a comment.