Click here to Skip to main content
15,069,133 members
Articles / Web Development / HTML
Article
Posted 21 Nov 2015

Stats

18.2K views
121 downloads
7 bookmarked

Taking audio stream from Twilio by SIP and sending to RTMP CDN for further delivery

Rate me:
Please Sign up or sign in to vote.
5.00/5 (1 vote)
21 Nov 2015CPOL11 min read
This is an article about integration SIP and RTMP protocols

Image 1

Introduction

As it is known, in the world exist CDN networks which are used for delivery of video streams to end users. Take an online broadcast of a football match for example.

Historically, protocol RTMP, developed by Adobe, at a certain time became very popular for delivery of Live video, and this protocol is supported by every respected Encoder – device or software application which captures video stream from camera and sends it to a server or CDN for further delivery.

Methods for further delivery of video to end users depend on the specific CDN and the technologies being used, which allow playing streams on a wide variety of devices and platforms: Apple HTTP Live Streaming, RTSP, MPEG-DASH, WebRTC, WebSockets, etc.

 

Image 2

Background

As well historically, protocol SIP (Session Initiation Protocol) is mostly known as a protocol for audio calls and telephony, and probably for this reason this protocol did not fall into the zone of interests of CDN providers and vendors of server software for CDN networks.

Although most people associate SIP only with telephony, the protocol SIP was originally planned and developed as a universal protocol for connection establishment with further delivery of data: audio, video, files, messages, etc.

As of today, the protocol definitely is not outdated, and a large number of software solutions and Internet services support SIP in one form or another. Mostly, those are software and equipment for video conferences, video chats, and communications.

As the result, in this technological junction – SIP and CDN – emerged an integration problem of using a SIP device or a telecommunication service as a source of video stream with further delivery of the stream to end users.

Image 3

The simplest example is a video conference or a webinar on educational or social subject, launched using software with SIP support and requiring broadcast to CDN network for a wide representation of the event.

In this article, for simplicity we restrict ourselves to capture of audio stream from a SIP conference and further broadcast of the stream using RTMP protocol. Strictly speaking, such action is rather a SIP request with subsequent conversion and redirection of the traffic than video capture. Image 4

On the scheme of capture and broadcast can be seen arrow «Command». Indeed, if there are three participants: SIP device, integration server and RTMP server, - then there has to be an interface or API which can be used to give a command to start capturing SIP stream and broadcasting, i.e. launch the case.

Another requirement: it must be possible to establish in parallel multiple call and broadcast sessions. I.e. establish pairs SIP Call 1 – RTMP Stream 1, SIP Call 2 – RTMP Stream 2, etc. For example, in case there are several webinars (rooms) at the same time. 

Image 5

As the SIP conference server/service we will be using Twilio service.

As the RTMP server, on the basis of which can be built CDN, Wowza Streaming Engine will be used.

Web Call Server 4 supporting conversion and broadcast of SIP to RTMP will be the integration server.

The command to initiate SIP call and redirect stream to RTMP will be sent to the server by REST/HTTP – probably, the most widespread in Internet method of software interaction based on HTTP.

This article describes how broadcast of a recorded message or a voice call session can be started by a simple REST/HTTP request.

Image 6

That would require

  • Registration in Twilio service and creation of SIP Domain
  • Installation and configuration of Wowza RTMP server
  • Installation and configuration of Web Call Server 4, broadcasting SIP as RTMP
  • Installation of REST Console extension for sending REST/HTTP requests manually from Google Chrome browser

Twilio

Twilio is a cloud-based telephony platform providing API which allows integration of phone services into web and mobile applications. To use Twilio, you need to sign up for an account.

When signed in, you can get a phone number which can be setup with a URL referring to an application. The application should contain a set of instructions telling Twilio what is to be done when an inbound call is received.

For example, play a message or connect to a conference call. Besides calling to a phone number associated with Twilio account, it is possible to connect to Twilio with SIP.

For that, it is necessary to create and configure a Twilio SIP Domain, which as well can be assigned a "Voice Url" of an application with instructions for Twilio. To create a Twilio SIP Domain, add a new SIP Endpoint

Image 7


SIP Endpoints List

Image 8


You can call to Twilio SIP Domain from a SIP device, e.g. a softphone. That would require to 1. Add the device public IP to the IP Access Control List of the domain:

Image 9


* Instead of ’192.168.1.5’, there should be the device public IP 2. Add credentials to the Credential List of the domain: 

 Image 10

 

3. Configure on the softphone a "without registration" account with the Twilio SIP Domain as the domain and the credentials added to the Credential List of the domain as user ID and password For example, such account in X-Lite softphone would look this way:

Image 11

4. Place a call to the Twilio SIP Domain (e.g. if the domain name is mytwiliodomain.sip.twilio.com, call to mytwiliodomain.sip.twilio.com)

Image 12

Thus, we called to Twilio from SIP softphone and, in fact, tested SIP side, receiving and playing audio stream from the call. Now it remains to make such call from WCS4 server and forward the received audio traffic as RTMP.

Image 13

Web Call Server 4

WCS4 server would require a Linux 64-bit server with installed Java. Linux console command java –version must display information about Java machine. If it does not, then Java can be installed with command yum install –y java for RedHat, and apt-get install java for Debian Linux distributions.

WCS4 server build can be downloaded here, and a trial license can be requested here.

The only change in the default WCS configuration necessary for streaming SIP as RTMP is codec restriction so that SIP session with Twilio SIP Domain could be established with audio only.

The codecs setting in the flashphoner.properties configuration file (in directory /usr/local/FlashphonerWebCallServer/conf) should be set to G.711 uLaw:

codecs = ulaw

The file can be edited, for example, using Midnight Commander (mc)

Image 14


Image 15

Wowza Streaming Engine

Wowza Streaming Engine can be downloaded from wowza.com. It would require a license; free Developer License can be requested.

Here can be found information on how to run Wowza Streaming Engine, and here - information on Wowza Streaming Engine Manager, in which it will be necessary to allow RTMP Publishing in 'Applications' | 'live' | 'Incoming Security' settings.

Image 16

Creation of REST / HTTP request for call initiation

Once WCS, Twilio SIP Domain and Wowza setup is complete, we can proceed to establishing a SIP session between WCS server and Twilio SIP Domain.

In this case, an HTTP / REST / POST request should be sent to the WCS server to initiate a SIP call. Here is an example of the REST request body in JSON format:

{ <br />
"callId":"R2fhkll5Sw3lK",<br />
"callee":"mytwiliodomain.sip.twilio.com",<br />
"rtmpUrl":"rtmp://my_wowza_server.com:1935/live",<br />
"rtmpStream":"streamName",<br />
"hasAudio":"true",<br />
"hasVideo":"true",<br />
"connection":{ <br />
      "sipLogin":"myusername",<br />
      "sipPassword":"MyPasswordPassword1",<br />
      "sipAuthenticationName":"myusername",<br />
      "sipDomain":"mytwiliodomain.sip.twilio.com",<br />
      "sipPort":"5060",<br />
      "sipRegisterRequired":"false",<br />
      "appKey":"callApp"<br />
   }}

The values of parameters of sub-object "connection" in the request are specified similarly to those in a softphone account:

  • sipRegisterRequired : false – i.e. without SIP registration
  • sipDomain : mytwiliodomain.sip.twilio.com - the Twilio SIP Domain
  • sipLogin and sipPassword- the credentials added to the Credential List of the Twilio SIP Domain

Among those parameters, there is also "appKey" parameter set to the name of the standard server-side application ('callApp') which will receive statuses of the SIP call and RTMP stream associated with the call.

Other parameters:

  • callee : mytwiliodomain.sip.twilio.com – callee’s name: the Twilio SIP Domain
  • callId : R2fhkll5Sw3lK – SIP Call ID, arbitrary unique identifier for the call
  • rtmpStream : streamName – name of the stream which will be published on the RTMP server
  • rtmpUrl : rtmp://my_wowza_server.com:1935/live – RTMP URL with address of the RTMP server to which the audio of the call will be transmitted
  • hasAudio : true – whether it will be indicated in SDP that WCS will be sending RTP audio traffic in the SIP call
  • hasVideo : true – whether it will be indicated in SDP that WCS will be sending RTP video traffic in the SIP call

Parameters "sendrecv" и "recvonly" in SIP

Parameters hasAudio and hasVideo are important. It is implied in the case that we do not send traffic to the SIP device, but only receive traffic from it.

If hasAudio and hasVideo are set to "true", then in SDP "sendrecv" (send and receive) will be indicated for audio and video. In that case some SIP devices will be awaiting traffic, and will disconnect the SIP session on timeout if there is no inbound traffic.

So, it will be necessary to increase RTP inactivity timer value on the SIP device, or set hasAudio and hasVideo to "false" to send in SDP "recvonly" (receive only). Then the SIP party will know that we are going only to receive video and the timeout should be ignored.

Posting REST / HTTP request from REST Console

Request URI for such REST call will be similar to http://my_wcs_server.flashphoner.com:9091/RESTCall/call, where my_wcs_server.flashphoner.com is the address of the WCS server.

To initiate a REST call, a REST client is required. It may be a Chrome extension like REST Console or Advanced REST client, or code attached to this article (requires installation of Allow-Control-Allow-Origin: * extension for Google Chrome browser).

Below is an example of the REST Console interface with the fields required for the REST request:

Image 17

If WCS server is started, Twilio receives SIP calls and all data is entered correctly, click "POST" button in the REST Console to send the request to the server.

As the result, SIP session between WCS and Twilio will be established and audio stream will be transmitted to the specified RTMP address.

Stream playback in RTMP player

The published RTMP stream can be played with Wowza example Flash RTMP player, e.g.

http://my_wowza_server.com/FlashRTMPPlayer/player.html

The values of the rtmpUrl and rtmpStream parameters specified in the REST request should be entered into the Server and Stream fields before starting the stream playback.


Image 18
 


When RTMP stream associated with the SIP call is played, the listener using the RTMP player will hear audio played by the application available by "Voice Url" assigned to the Twilio SIP Domain.

Congratulations! If you hear audio, this means you have successfully deployed and tested the 'SIP to RTMP' case with Twilio.

Image 19

Broadcast of a video stream, additional information

Not only audio, as in case of call with Twilio, but video as well can be streamed using the "SIP as RTMP" capability of WCS4.

The full list of codecs supported for the feature is

  • Audio: G.711 (allaw or ulaw) or Speex 16 kHz
  • Video: H.264

The image below illustrates call flow communication paths

Image 20 1. Signaling (black lines)

  • HTTP / REST / POST request to WCS server
  • SIP session between WCS server and SIP Endpoint (e.g. Twilio SIP Domain)
  • RTMP connection between WCS server and RTMP server

2. Streaming (red lines)

  • RTP media from SIP Endpoint to WCS server
  • RTMP stream from WCS server to RTMP server

Video can be streamed, for example, for a video SIP call established with a softphone, e.g. Bria. In that case, the values of REST call parameters.

sipDomain and sipPort - will be the address and port of the SIP server to which the softphone is registered
sipLogin and sipAuthenticationName and sipPassword – the credentials of an account on the SIP server
callee – SIP user name with which the softphone is registered

Conclusion

In this way, we were able to integrate Twilio service and RTMP server, redirecting SIP audio stream to RTMP for further delivery by any technological means available.

Using the code

Download sip-as-rtmp-html-js-example.zip

  1. Make sure you have all necessary background installed: Wowza, Web Call Server, Twilio.
  2. Open html page in your web browser and manage SIP/RTMP calls and manage your SIP / RTMP calls. 

Note,

The HTML page won't work in crossdomain mode (i.e. if you open sip-as-rtmp.html page locally or if it is loaded by HTTP from a different domain).

To avoid the crossdomain issue you may add Chrome extension Allow-Control-Allow-Origin: * or use REST Console or any other REST client manually.

Links

Allow-Control-Allow-Origin: * - a Chrome browser extension to resolve crossdomain issue for sip-as-rtmp.html page

REST Console - a Chrome browser extension for manual REST/HTTP testing

Web Call Server - integration streaming media server 

Wowza Streaming Engine - streaming media server

Twilio - cloud telephony platform

X-Lite - free SIP softphone for testing 

 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

MonkeySeeMonkeyWrite
United States United States
No Biography provided

Comments and Discussions

 
-- There are no messages in this forum --