WebRTC WHIP & WHEP Tutorial: Build a live Streaming App

[Lomanu4]

This article was originally published on the Metered Blog:

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

WHIP (WebRTC-HTTP Ingestion Protocol) and WHEP (WebRTC-HTTP Egress Protocol) are protocols that are designed to streamline signalling in WebRTC with the help of standard HTTP methods

• 
  Definition of WHIP: WHIP simplifies how client devices send media streams to the server.
  
  • It replaces the complex signalling mechanism that is required with simple HTTP GET requests, thus making it easier to ingest media into servers
• 
  Definition of WHEP: WHEP protocol is used for delivering media streams from servers to clients. It uses the HTTP protocol to handle signalling for media consumption thus enabling client devices to receive media streams without complex setups
  

Roles in Simplifying WebRTC Signalling

• 
  Ease of Implementation: WHEP and WHIP uses HTTP protocols thus these protocols reduce complexity that is associated with
  
  
• 
  Stateless Communication: This is because the HTTP is stateless protocol, the server does not need to maintain ongoing session information between requests.
  
  
• 
  Improved Compatibility: Since the HTTP has universal compatibility, using HTTP for signalling is the best for compatibility across platforms and devices
  
  
• 
  Fast development: Developers can implement WebRTC apps more efficiently, because they do not need to take into account intricate details of traditional signalling methodologies
  

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

How does WHIP work

How does WHIP handle Media Stream Ingestion


WHIP protocol has revolutionized how media streams can be sent to the servers from client devices by using the HTTP methods for signalling

Traditional to setup the WebRTC you need to setup complex signalling mechanism using web sockets or other protocols. With WHIP this process becomes simple by using the HTTP protocol for signalling and starting a WebRTC session

• 
  HTTP POST Request: Here the client devices sends an HTTP POST request with the SDP or the session description protocol offer in the body to the WHIP endpoint
  
  
• 
  Server Response: The Media server then processes the SDP offer and responds with 200 status code including the SDP answer in the request body
  
  
• 
  ICE Candidate Exchange: The WHIP protocol supports the ICE protocol by allowing the client to send additional HTTP PATCH requests whenever new ICE candidates become available
  
  
• 
  Connection Establishment: Once the SDP exchange is complete then a peer to peer connection is established enabling the client to stream the media to the server
  

Benefits over traditional Ingestion Methods

• 
  Simplicity: By using the WHIP methods the WHIP reduces the need for persistent connections and signalling servers.
  
  
• 
  Ease of Implementation: Developers can use HTTP which has universal compatibility to speed up the development process
  
  
• 
  Scalability: The stateless HTTP requests allow the servers to handle multiple connection requests simultaneously thus easily managing a large number of connections.
  
  
• 
  Firewall and Proxy Friendly: HTTP is firewall friendly, almost all types of firewall allow HTTP traffic
  
  
• 
  Cost Effective: The simplified signalling through HTTP reduces the costs associated with adding a signalling server
  

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

How does WHEP work


The WHEP protocol simplifies the process of delivering media from server to the client devices.

Thus the WHEP protocol enables you to use HTTP to setup signalling for receiving media from server the client devices.

How does WHEP work in media streaming

• 
  HTTP GET Request: The client requests a media stream by sending an HTTP GET request to the servers WHEP endpoint
  
  
• 
  SDP Exchange: The server responds with the SDP offer in HTTP response, the client then sends back the SDP answer in the subsequent POST request
  
  
• 
  Media Reception: Once the connection is established the media stream is received over the established WebRTC connection. NOTE: Many times you need a
  
  

  Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

  
  
  to establish a WebRTC connection
  
  
• 
  Support for ICE: WHEP allows the exchange of ICE Candidates through additional HTTP patch requests thus enabling better connectivity
  

Advantages in client side streaming

• 
  Simplified Client Implementation: Use of HTTP requests hence reducing the need for complex signalling mechanisms
  
  
• 
  Improved Compatibility: Universal support for the HTTP protocol ensures improved compatibility across devices
  
  
• 
  Enhanced Scalability: Because the HTTP is a stateless protocol, this improves the scalability of the servers and with small resources you can scale to a very large number of users
  
  
• 
  Better Network Traversal: Because you can do signalling with HTTP and do not need web sockets or other mechanisms, this improves NAT traversal for connectivity. Once the connection is established you do need
  
  

  Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

  
  
  
  
  
• 
  Reduced Latency: Signalling using HTTP can lead to faster connections thus enhancing user experience.
  

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

Synergy between WHIP and WHEP

Combining Both Protocols for Efficient End-to-End Communication:


By combining WHIP and WHEP the developers can create a comprehensive signalling solution for WebRTC

The combination simplifies the ingestion and delivery of media streams, ensuring a smoother implementation of WebRTC

• 
  Unified Signalling Approach: Using the HTTP for both ingestion and delivery creates a consistent signalling methodology
  
  
• 
  Reduced Complexity: Developers need to deal with only HTTP protocol, thus reducing the learning curve and maintenance of code
  
  
• 
  Enhanced Performance: Streamlining the code with a single protocol for signalling leads to quicker connection times and lower latency when you are transmitting media
  

Use-Cases Showcasing Improved Performance

• 
  Live Streaming Platforms:
  
  
• 
  Interactive Applications
  
  
• 
  Scalable Architecture
  

Building The Live Streaming APP


Implementing the WHIP and WHEP in your WebRTC app is quite easy. In this section, we will be setting up a WHIP server and integrating it in your application using modern technologies like node and docker

Here we are going to use Metered.ca TURN server service for NAT traversal

Setting Up the WHIP Server


Pre-requisites and Environment Setup:

• 
  Node.js and NPM: Make sure you have the latest Node and nvm installed
  
  
• 
  Metered.ca Account: Create an free account on
  
  

  Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

  
  
  
  
  
• 
  Public IP Address: This is required for the server to be accessible on the internet. If you are using any cloud provider for your application you get a free public IP address with that
  
  
• 
  WebRTC Media Server: We need a media server such as GStreamer or Janus that has WHIP support
  

Step up Step Configuration Using Node.Js and GStreamer

1. 
   Install GStreamer with WHIP Support
   
   
2. 
   Set Up the WHIP Server with GStreamer
   
   1. Create a GStreamer pipeline that listens to the WHIP connections like so

gst-launch-1.0 whipserversrc name=whip \
! queue ! videoconvert ! autovideosink

The above command starts a WHIP server that accepts incoming media streams and displays them

1. 
   Configure the Server to Use
   
   

   Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

   
   
   
   
   1. Modify the GStreamer pipeline to use TURN server. This is important because NAT routers and firewalls block connections

gst-launch-1.0 whipserversrc name=whip ice-server="turn://YOUR_USERNAME:YOUR_CREDENTIAL@relay.metered.ca:80" \
! queue ! videoconvert ! autovideosink

1. 
   Set Up a Reverse Proxy with Node.JS (Optional):
   
   1. If you want to manage HTTP endpoints or add any other additional logic, you can set up a simple express js server

const express = require('express');
const httpProxy = require('http-proxy');
const app = express();
const proxy = httpProxy.createProxyServer();

app.post('/whip', (req, res) => {
proxy.web(req, res, { target: 'http://localhost:PORT_WHERE_GSTREAMER_IS_RUNNING' });
});

app.listen(3000, () => {
console.log('Proxy server running on port 3000');
});
Implementing WHIP in your Application


Code Snippets for Integrating WHIP:

On the client side, you can capture media streams with the help of RTCPeerConnection and use HTTP requests to handle the signalling that is required to establish a connection

1. 
   Capture Media Streams:
   
   1. You can capture media streams like

const mediaStream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });

1. Create an RTCPeerConnection:

You can create an RTCPeerConnection with the help of Metered TURN Servers


var myPeerConnection = new RTCPeerConnection({
iceServers: [
{
urls: "stun:stun.relay.metered.ca:80",
},
{
urls: "turn:global.relay.metered.ca:80",
username: "your-username",
credential: "your-credential",
},
{
urls: "turn:global.relay.metered.ca:80?transport=tcp",
username: "your-username",
credential: "your-credential",
},
{
urls: "turn:global.relay.metered.ca:443",
username: "your-username",
credential: "your-credential",
},
{
urls: "turns:global.relay.metered.ca:443?transport=tcp",
username: "your-username",
credential: "your-credential",
},
],
});

1. Add Media Tracks to the Connection:

mediaStream.getTracks().forEach((track) => {
pc.addTrack(track, mediaStream);
});

1. Create and Send SDP Offer to WHIP Server:

const offer = await pc.createOffer();
await pc.setLocalDescription(offer);

const response = await fetch('http://YOUR_SERVER_IP:3000/whip', {
method: 'POST',
headers: { 'Content-Type': 'application/sdp' },
body: pc.localDescription.sdp,
});

const sdpAnswer = await response.text();
await pc.setRemoteDescription({ type: 'answer', sdp: sdpAnswer });
Handling HTTP Requests and Responses for Signalling

• 
  Client Side:
  
  • 
    HTTP POST Request:
    
    • URL:
      
      

      Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

      
      
    • Headers: { 'Content-Type': 'application/sdp' }
    • Body: SDP offer as plain text
  • 
    Expecting Response:
    
    • Status: 201 Created
    • Headers: Location header with resource URL
    • Body: SDP answer as plain text
• 
  Server Side:
  
  • 
    Receive SDP Offer:
    
    • Read the SDP from the req.body
    • Create WebRTC endpoint and set the remote description
  • 
    Generate SDP Answer
    
    • An SDP answer from the server WebRTC endpoint
    • Send SDP answer back in the res.body
    • Using
      
      

      Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

      
      
      TURN server service

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

Using Metered.ca TURN server service

Purpose of TURN Server


Facilitates media traversal through NAT and firewall when direct peer to peer connection is not possible

Incorporate TURN Server in ICE Server Configuration


Here is a TURN Server credential and ICE Server


var myPeerConnection = new RTCPeerConnection({
iceServers: [
{
urls: "stun:stun.relay.metered.ca:80",
},
{
urls: "turn:global.relay.metered.ca:80",
username: "e13b9bsdfdsfsdfb0676cc5b6",
credential: "dedewdewfer+gq5iT",
},
{
urls: "turn:global.relay.metered.ca:80?transport=tcp",
username: "e13bdfdsfds6b0676cc5b6",
credential: "dewfrefre+gq5iT",
},
{
urls: "turn:global.relay.metered.ca:443",
username: "e13b9fsdfdsfsd86b0676cc5b6",
credential: "csdfwefeer+gq5iT",
},
{
urls: "turns:global.relay.metered.ca:443?transport=tcp",
username: "e13b9dsfsdfe6b0676cc5b6",
credential: "sdfewtrererer+gq5iT",
},
],
});
Deploying a WHEP client


Having a WHIP client allows your app to receive media streams from the server using HTTP signalling.

Integrating WHEP on the client side

• 
  Basic understanding of WebRTC API in Javascript
  
  
• 
  Media server that supports WHEP GStreamer Janus or any other
  
  
• 
  
  
  

  Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

  
  
  TURN server credentials
  

Step by step WHEP integration in your App.

1. 
   Initialize the RTCPeerConnection
   
   1. Create a RTCPeerConnection instance with the ICE server that has
      
      

      Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

      
      
      turn servers

var myPeerConnection = new RTCPeerConnection({
iceServers: [
{
urls: "stun:stun.relay.metered.ca:80",
},
{
urls: "turn:global.relay.metered.ca:80",
username: "e13b9bsdfdsfsdfb0676cc5b6",
credential: "dedewdewfer+gq5iT",
},
{
urls: "turn:global.relay.metered.ca:80?transport=tcp",
username: "e13bdfdsfds6b0676cc5b6",
credential: "dewfrefre+gq5iT",
},
{
urls: "turn:global.relay.metered.ca:443",
username: "e13b9fsdfdsfsd86b0676cc5b6",
credential: "csdfwefeer+gq5iT",
},
{
urls: "turns:global.relay.metered.ca:443?transport=tcp",
username: "e13b9dsfsdfe6b0676cc5b6",
credential: "sdfewtrererer+gq5iT",
},
],
});

1. Handle incoming Media Tracks

Setup an event listener to revive remote tracks from the server


pc.addEventListener('track', (event) => {
const [remoteStream] = event.streams;
// Attach the remote stream to a video element
const remoteVideo = document.getElementById('remoteVideo');
remoteVideo.srcObject = remoteStream;
});

1. Send an HTTP GET Request to the WHEP server:

Send a GET request to the server


const whepServerEndpoint = 'http://YOUR_SERVER_IP:3000/whep'; // Replace with your server's WHEP endpoint

const response = await fetch(whepEndpoint, {
method: 'GET',
headers: {
Accept: 'application/sdp',
},
});

const sdpOffer = await response.text();

1. Remote description with the received SDP offer

await pc.setRemoteDescription({
type: 'offer',
sdp: sdpOffer,
});

1. Create and send the SDP answer

Create an SDP answer and send it to the server through a HTTP POST request


const sdpAnswer = await pc.createAnswer();
await pc.setLocalDescription(sdpAnswer);

await fetch(whepEndpoint, {
method: 'POST',
headers: {
'Content-Type': 'application/sdp',
},
body: pc.localDescription.sdp,
});

1. Handle ICE Candidate Exchange (Optional):

If you need to send ICE candidates separately then handle the icecandidate event


pc.addEventListener('icecandidate', (event) => {
if (event.candidate) {
// Send the candidate to the server through HTTP PATCH request
fetch(whepEndpoint, {
method: 'PATCH',
headers: {
'Content-Type': 'application/trickle-ice-sdpfrag',
},
body: event.candidate.candidate,
});
}
});
Handle Media streams on the front-end

1. create a video element in HTML

<video id="remoteVideo" autoplay playsinline></video>

1. Attach remote stream to the video element

when a track event is a fired attach the recieved stream to the video element

1. 
   Handling the Media Stream Event
   
   1. Connection state change

pc.addEventListener('connectionstatechange', () => {
if (pc.connectionState === 'connected') {
console.log('WebRTC connection established');
} else if (pc.connectionState === 'failed' || pc.connectionState === 'disconnected') {
console.error('WebRTC connection failed or disconnected');
}
});

b. Negotiation needed


pc.addEventListener('negotiationneeded', async () => {
try {
const offer = await pc.createOffer();
await pc.setLocalDescription(offer);
// Send new SDP offer to the server if needed
} catch (error) {
console.error('Error during renegotiation:', error);
}
});

1. Full example code

async function startWHEPClient() {
var pc = new RTCPeerConnection({
iceServers: [
{
urls: "stun:stun.relay.metered.ca:80",
},
{
urls: "turn:global.relay.metered.ca:80",
username: "e13b9bsdfdsfsdfb0676cc5b6",
credential: "dedewdewfer+gq5iT",
},
{
urls: "turn:global.relay.metered.ca:80?transport=tcp",
username: "e13bdfdsfds6b0676cc5b6",
credential: "dewfrefre+gq5iT",
},
{
urls: "turn:global.relay.metered.ca:443",
username: "e13b9fsdfdsfsd86b0676cc5b6",
credential: "csdfwefeer+gq5iT",
},
{
urls: "turns:global.relay.metered.ca:443?transport=tcp",
username: "e13b9dsfsdfe6b0676cc5b6",
credential: "sdfewtrererer+gq5iT",
},
],
});

pc.addEventListener('track', (event) => {
const [remoteStream] = event.streams;
const remoteVideo = document.getElementById('remoteVideo');
remoteVideo.srcObject = remoteStream;
});

pc.addEventListener('connectionstatechange', () => {
if (pc.connectionState === 'connected') {
console.log('WebRTC connection established');
}
});

const whepEndpoint = 'http://YOUR_SERVER_IP:3000/whep';

try {
const response = await fetch(whepEndpoint, {
method: 'GET',
headers: {
Accept: 'application/sdp',
},
});

const sdpOffer = await response.text();

await pc.setRemoteDescription({
type: 'offer',
sdp: sdpOffer,
});

const sdpAnswer = await pc.createAnswer();
await pc.setLocalDescription(sdpAnswer);

await fetch(whepEndpoint, {
method: 'POST',
headers: {
'Content-Type': 'application/sdp',
},
body: pc.localDescription.sdp,
});

} catch (error) {
console.error('Error occoured during the WHEP connection:', error);
}
}

// Start the WHEP client
startWHEPClient();

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

1. 
   API: TURN server management with powerful API. You can do things like Add/ Remove credentials via the API, Retrieve Per User / Credentials and User metrics via the API, Enable/ Disable credentials via the API, Retrive Usage data by date via the API.
   
   
2. 
   Global Geo-Location targeting: Automatically directs traffic to the nearest servers, for lowest possible latency and highest quality performance. less than 50 ms latency anywhere around the world
   
   
3. 
   Servers in all the Regions of the world: Toronto, Miami, San Francisco, Amsterdam, London, Frankfurt, Bangalore, Singapore,Sydney, Seoul, Dallas, New York
   
   
4. 
   Low Latency: less than 50 ms latency, anywhere across the world.
   
   
5. 
   Cost-Effective: pay-as-you-go pricing with bandwidth and volume discounts available.
   
   
6. 
   Easy Administration: Get usage logs, emails when accounts reach threshold limits, billing records and email and phone support.
   
   
7. 
   Standards Compliant: Conforms to RFCs 5389, 5769, 5780, 5766, 6062, 6156, 5245, 5768, 6336, 6544, 5928 over UDP, TCP, TLS, and DTLS.
   
   
8. 
   Multi‑Tenancy: Create multiple credentials and separate the usage by customer, or different apps. Get Usage logs, billing records and threshold alerts.
   
   
9. 
   Enterprise Reliability: 99.999% Uptime with SLA.
   
   
10. 
    Enterprise Scale: With no limit on concurrent traffic or total traffic. Metered TURN Servers provide Enterprise Scalability
    
    
11. 
    5 GB/mo Free: Get 5 GB every month free TURN server usage with the Free Plan
    
    
12. 
    Runs on port 80 and 443
    
    
13. 
    Support TURNS + SSL to allow connections through deep packet inspection firewalls.
    
    
14. 
    Supports both TCP and UDP
    
    
15. 
    Free Unlimited STUN
    

You can consider some of our other articles:

1. 
   
   
   

   Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

   
   
   
   
   
2. 
   
   
   

   Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

   
   
   
   
   
3. 
   
   
   

   Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

   
   
   
   
   
4. 
   
   
   

   Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

   
   
   
   

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.