How to implement web to web voice calls using Silverlight microphone access

After starting your Visual Studio, you will need to create a new project for your webphone server. As the server is only a service provider that runs in the background it can be a console application that means that it will not have a GUI.

You can create a new project by choosing the File->New Project menu from the menu bar (Figure 3). This menu opens the New Project window where you can choose the project type from a template list.

Figure 3 - File->New Project menu helps you to create a new project

You should choose Console Application from the list on the New Project window shown in Figure 4. You can rename your project and change the project location according to your wish on the bottom panel of this window. You will be able to do these changes later, but it is easier to do them before starting the programming stage.

Figure 4 - You can choose the project type and set the project properties on the New Project window

Step 2. How to get background support from Ozeki VoIP SIP SDK?

The webphone server application needs the background support of the MediaGateway SDK that is part of Ozeki VoIP SIP SDK. You will need to register this SDK to your server project in order to use its functionality support.

You can register your SDK to your project on the Solution Explorer panel that is usually on the right side of the Visual Studio window. You need to right click on the References label and choose Add Reference from the appearing list (Figure 5).

Figure 5 - You can register your Ozeki VoIP SIP SDK on the Solution Explorer panel

You can browse your hard drive for Ozeki VoIP SIP SDK .dll file and press OK when you have found it (Figure 6). If you have installed Ozeki VoIP SIP SDK without interaction, the .dll will be in "C:\Program Files\Ozeki\SDK" folder.

Figure 6 - You can choose Ozeki VoIP SIP SDK .dll by browsing your file system

After registering Ozeki VoIP SIP SDK you will be able to use the MediaGateway SDK that provides the background support for the webphone server application.

Step 3. How to implement the server functionality?

You can add new classes to your project on the Solution Explorer panel. You should right click on the project name and choose Add->Class from the floating menu that appears or you can simply press Shift+Alt+C key combination to create a new class (Figure 7).

In the Add New Item window, you can specify the item you want to add to your project. In this case it will be a Class but you can also choose for example Interface, Windows Form, UML diagrams, XML files, etc. You can specify the Name of the new item and press OK to create it.

Figure 7 - You can define a new class on the Solution Explorer panel

After having your brand new class, you will need to be able to reach the support Ozeki VoIP SIP SDK provides and this is more comfortable without using namespace labels all the time, so you will need to add a new line (Code 1) in the using section of the class.

using Ozeki.MediaGateway;

This single code line will save you a lot of tying and also allows you to inherit the methods and tools that are implemented in Ozeki.MediaGateway.MediaGateway class. You have to specify this inheritance by expanding the class definition line to look like as the one in Code 2.

class Web2WebGateway : MediaGateway

You will need two fields in your class, one for the clients and the other for counting the clients. As for the clients, you can use the System.Collections.Generic.Dictionary@lt;TKey, TValue> class. This class is a hash table that stores the values with a specific key for identification.

The Dictionary class will be more comfortable to use if you append another line to your using section (Code 3).

using System.Collections.Generic;

The Dictionary class is a generic class, so you can specify the type of the key and the value of the objects that are stored in it. In this case the key type will be IClient that is a built-in type in Ozeki VoIP SIP SDK and the value type is MyClient that is a class you will have to define yourself (Code 4).

private Dictionary<IClient,MyClient> Clients;

MyClient class is another self-defined class that you should add to your project the same way as you did in case of the Web2WebGateway class that is the main class of this project.

The MyClient class is basically for defining the invoke methods for performing the client-side methods. The server has to be able to make the clients start some of their operations in order to ensure the proper work of the webphone.

The MyClient class stores three basic information fields that are Name, IsBusy and RemoteParty. These give you the basic knowledge about the state of the represented client and the other client it is connected to if there is any.

MyClient class also implements the client-side method calls that are OnStartPlay, OnSetReadyStatus, OnCallRequest, OnInCall, OnPlayRemoteStream, OnCallStop. These methods define the invoke calls for the clients that were shown in Figure 2 above.

The client-side solution has to have a proper method for all these invoke methods that will be called when the server calls these methods with the specific client ID.

The OnStartPlay method implements an asynchronous message from the server to the remote client about that the other client started sending a voice stream. On client side the OnPlay method will be called to perform the proper functionality.

public void OnStartPlay(string remotpartyId)
{
    Client.InvokeMethod("OnPlay", remotpartyId);
}

The OnSetReadyStatus method invokes the client side method OnSetReadyStatus that will make the basic setting for the client to be ready for incoming and outgoing calls (Code 6).

public void OnSetReadyStatus(bool isReady, string name)
{
    try
    {
        Client.InvokeMethod("OnSetReadyStatus", isReady, name);
    }
    catch (Exception)
    {}
}

When a call request comes to the server from one of the clients, it will send an OnCallRequest to the other client with the information about the caller and sets both clients busy. This is done because all new clients that connect to the server can only call another not busy client, and in order to avoid collision, the server instantly sets both connected parties busy (Code 7).

public void OnCallRequest(string requestOwner)
{
    Console.WriteLine("Call request received from {0} to {1}",requestOwner, Name);
    RemoteParty.IsBusy = true;
    IsBusy = true;
    Client.InvokeMethod("OnCallRequest", requestOwner);
}

The OnIncall invoke method makes both clients start their OnIcall method that will ensure the synchronization of the clients to the call (Code 8).

public void OnInCall()
{
    Console.WriteLine("Sends 'start publishing' sign to the clients.");
    Client.InvokeMethod("OnInCall");
    RemoteParty.Client.InvokeMethod("OnInCall");
}

The OnPlayRemoteStream method is a message to the clients that the other party is ready for sending voice streams to them. The voice call is established properly when both clients get the OnPlayRemoteStrem message from the server (Code 9).

public void OnPlayRemoteStream()
{
    Console.WriteLine("PlayRemoteStream - client Name : {0} starts to play remoteStream: {1}", RemoteParty.Name, Name);
    RemoteParty.Client.InvokeMethod("OnPlayRemoteStream", Name);
}

The server calls the OnCallStop method when one of the clients sent a CallStop message to the server. In this case both clients will be free to start and accept new calls as their IsBusy flag will be set to false (Code 10).

public void OnCallStop()
{
    IsBusy = false;
    RemoteParty.IsBusy = false;
    Client.InvokeMethod("OnCallStop");
}

After writing these essential client invocation methods and having some getters and setters for the basic fields of MyClient class you can return to your main server class and finish it properly.

The main server functionality contains the actions that have to be done when the server starts, when a client connects or disconnects to or from the server and when a stream publishing starts. These methods are written in MediaGateway SDK and you need to override them for defining your own server behavior.

As the server is a console application, it will use the Console.WriteLine method to write some notifications or information about the current actions that are in process.

You will need to override the OnStart, OnClientConnect, OnClientDisconnect and OnStreamPublishStart methods of the MediaGateway.

The OnStart method defines all the activities that invoke when the webphone server starts. In this case the server only initializes the Clients table and writes a notification onto the console about the start (Code 11).

public override void OnStart()
{
    Clients = new Dictionary<IClient, MyClient>();
    Console.WriteLine("Web2Web Gateway started.");
}

When a client sends a connection request to the server, the OnClientConnect method will be called. If the client has not been added to the Clients table before, the server adds it to the table and calls the NotifyClientsAboutTheirCallStatus() method that also has to be written in this class (Code 12).

public override void OnClientConnect(IClient client, object[] parameters)
{
    Console.WriteLine( "{0} client connected to the server.",client.RemoteAddress);
    if (!Clients.ContainsKey(client))
    {
        Clients.Add(client, new MyClient(client, string.Format("client{0}", clientCounter++)));
        NotifyClientsAboutTheirCallStatus();
    }
}

The NotifyClientsAboutTheirCallStatus() method should contain a repetitive instruction that is a foreach block seen in Code 13 that sets the ready status of all the connected clients.

foreach (KeyValuePair<IClient, MyClient> keyValuePair in Clients)
                {
                    keyValuePair.Value.OnSetReadyStatus(isReady, keyValuePair.Value.Name);
                }

When a client is disconnected from the server it has to be removed from the Clients table and all clients has to be notified about the status change. If the disconnected client had an established call line at the time of disconnection, the remote connected client has to get an OnCallStop message in order to stop the call properly. The OnClientDisconnect method that defines this functionality is seen in Code 14.

public override void OnClientDisconnect(IClient client)
{
    Console.WriteLine("{0} client disconnected from the server.", client.RemoteAddress);
    if (Clients.ContainsKey(client))
    {
        MyClient disconnectedClient = Clients[client];
        if (disconnectedClient.IsBusy && disconnectedClient.RemoteParty!=null)
            disconnectedClient.RemoteParty.OnCallStop();
        Clients.Remove(client);
        NotifyClientsAboutTheirCallStatus();
    }
}

The OnStreamPublisStart method is for performing the voice stream start towards the connected clients. It only contains a notification line written on the console and a method call that starts publishing the stream that is represented by the mediaStream object to the client (Code 15).

public override void OnStreamPublishStart(IClient client, IMediaStream mediaStream)
{
    Console.WriteLine("client : {0} publish his stream : {1}",client.RemoteAddress,mediaStream.Name);
    base.OnStreamPublishStart(client, mediaStream);
}

You will need to write Call, ChangeToIncall, PlayRemoteStream, CallStop methods in your server class that are mainly used for notifying the clients about other side actions and for setting some client data on the server side.

The last step for the server implementation is to create a configuration file that will store the basic information about the server such as the IP address, the port number it uses, etc. For this purpose you will need to add a New Item (as it was shown above in Figure 7) to your project and this time it should be an XML file with the name of App.config.

The App.config XML file is shown in Code 16. You can use similar setting for your server but note that you have to set your proper IP address and port number in order to have a properly operable server.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <sectionGroup name="mediaGateway">
      <section name="mediaGatewayConfig" type="Ozeki.MediaGateway.Config.MediaGatewayAppConfig, VoIPSDK"/>
    </sectionGroup>
  </configSections>
  <mediaGateway>
    <mediaGatewayConfig>
      <providers>
        <add type="silverlight" localIPAddress="0.0.0.0" listenedPort="4502" serviceName="Web2WebServer"/>
        <add type="flash" serviceName="Web2WebServer"/>
      </providers>
    </mediaGatewayConfig>
  </mediaGateway>
</configuration>

After having a fully configured server-side application, it is time to start making the client side solution with Silverlight.

Step 4. How to implement the client?

The Silverlight client-side application needs the MediaGateway_SLClient.dll to be able to reach the support Ozeki provides for a Silverlight webphone client.

You will need to create a new Visual studio solution and have a new Silverlight Application for your client. You will need to be sure that the Silverlight version is correct that is Silverlight 4 and the web project type is ASP.NET Web Application Project (Figure 8).

Figure 8 - You have to make sure your setting are proper for the client

You will need to register the Ozeki MediaGateway_SLClient.dll to your project on the Solution Explorer panel.

The Silverlight Visual Studio project has two main parts. One for the application itself that contains the GUI and the background functionalities and another that is a test html page that will contain your Silverlight application to be able to be tested. You will not need to do anything with this html, it is only for testing purposes.

First you will need to create a graphical user interface (GUI) for your webphone. As this example is a click-to-call webphone, you will only need to have two buttons and some notification textboxes.

You can create the GUI as easily as in case of a Windows Forms Application only that there are different GUI elements you can choose from this time. After some properties to be set your GUI should look similar to the one in Figure 9.

Figure 9 - A simple click-to-call webphone GUI using Silverlight

The client has to have the following functionalities implemented:

• make connection to the server
• send notification about a connection state change
• start and accept a call
• stop a call
• play incoming audio stream
• set the client status
• detect an incoming call
• establish the MediaStreamSender
• release the used tools

The above mentioned functions will be implemented in separate methods, some of which will contain the button handlers and the textboxes' settings and the others will be the actual client-server communication and setting methods.

The first step as in case of every class is to add the useful namespace specifications to the using section of the code (Code 17).

The using.System lines are used for the C# built-in tools and the last two using lines are the ones that are namespaces provided by the Ozeki VoIP SIP SDK. These using commands are essential for the client application if you do not want to label all the Ozeki MediaGateway tools with the full namespace path.

using System;
using System.Diagnostics;
using System.Windows;
using System.Windows.Controls;
using Ozeki.MediaGateway;
using Ozeki.MediaGateway.SilverlightService;

You will need to define some fields in your class that are used for the connection and the communication, and some of them are for storing the local settings of the client (Code 18).

The MediaConnection, MediaStreamSender, MediaStreamReceiver, AudioPlayer and Microphone classes are provided by the Ozeki VoIP SIP SDK. The other tools are mainly for standard setting storages within the class.

The MediaConnection object is the one that is for establishing a connection between the client and the server. It needs at least the server address as parameter, but you can also add a value for the connection timeout as an optional second parameter.

The MediaStreamSender and MediaStreamReceiver objects are for sending and receiving media data, in this case audio streams to and from the server. The AudioPlayer object is the tool that can actually play the received audio stream.

private MediaConnection connection;
private MediaStreamSender streamSender;
private MediaStreamReceiver streamReceiver;
private AudioPlayer audioPlayer;
private Microphone microphone;
private string clientID;
private string IncomingCallOwner;
private bool callProcess;

The connection_ConnectionStateChanged method is mainly a switch block that initializes the AudioPlayer in case of a succeeded connection to the server and sets the information textbox Text to the proper state that can be Online for a succeeded connection or Connection failed (Code 19).

switch (e.Item)
{
case ConnectionState.Success:
    txtboxInfo.Text = "Online";
    audioPlayer = new AudioPlayer();
    connection.Client = this;
    break;
default:
    InvokeGUIThread(() => {
        txtboxInfo.Text = "Connection failed.";
        btnStopCall.IsEnabled = btnCall.IsEnabled = false;
    });
    break;
}

The Call button has double functionality that needs to be implemented in its handler method. You can get the default handler method of the button by double clicking on it on the Design tab.

You will need to check if there is any incoming call as in that case the call button will accept the call. In case of no incoming call, the client will start a call and send a Call message to the server that will forward the call request to an available connected client (Code 20).

You will also have to check if the Silverlight has the right to access the microphone that is connected to your computer. It is essential as the microphone is the device that will capture your speaking for the audio stream that will be delivered to the other party.

if (Microphone.GetPermissionToMicrophone())
{
ReleaseStreams();
callProcess = true;
if (!string.IsNullOrEmpty(IncomingCallOwner))
{
    connection.InvokeOnConnection("ChangeToIncall");

    IncomingCallOwner = "";
    btnCall.IsEnabled = false;

}
else
{
    connection.InvokeOnConnection("Call", clientID);
    txtboxInfo.Text = "Outgoing call progress.";
    btnCall.IsEnabled = false;
}
}
else
{
txtboxInfo.Text = "Please, add permission to access microphone.";
}

The Stop Call button has a simple task to perform that is stopping an existing call. If there is no call established, the Stop Call button will do nothing. In case of an existing call the client will stop the call by invoking the connection's CallStop method (Code 21).

Stopping a call also means that you have to free the Streams you have used. This is done by the method call ReleaseStreams(). This method has to be written in this class too. After stopping a call you will be able to start or accept a new one, so you will need to be able to use the Call button, this is why it is enabled.

if (callProcess)
{
txtboxInfo.Text = "Call stopped, ready to call.";
connection.InvokeOnConnection("CallStop");
ReleaseStreams();
btnCall.IsEnabled = true;
}

You will also need to write some helping method for the main purposes of the client that can enable, disable or simply initiate the fields in case of need.

The call establishment needs the basic initializations of the used tools that should be written in the OnInCall method (Code 22).

In order to be able to establish and use a call, you need to set the microphone and attach it to the MediaSender. Ozeki MediaGatewy SDK gives you the support for these tasks, so you will only need to call some methods as it is shown in Code 22.

The MediaStreamSender object also has to be signed up to the StreamStateChanged event as it is essential to know if the stream can be sent or not.

The last step in case of a call is to publish the client ID to indicate that the client is ready for the media sending and receiving.

txtboxInfo.Text = "Incall";
    microphone = Microphone.GetMicrophone();
    streamSender = new MediaStreamSender(connection);

    try
    {
        streamSender.AttachMicrophone(microphone);
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.Message);
    }

    streamSender.StreamStateChanged += new EventHandler<GenericEventArgs<Ozeki.MediaGateway.SilverlightService.StreamState>>(streamSender_StreamStateChanged);

    streamSender.Publish(clientID);

In order to be able to receive audio data, you will need to write the OnPlayRemoteStream method (Code 23) that will initiate and attach the MediaStreamReceiver object to the connection. You will also need to start the stream playing to be able to hear what the other side tells.

streamReceiver = new MediaStreamReceiver(connection);
streamReceiver.StreamStateChanged += new EventHandler<GenericEventArgs<StreamState>>(streamReceiver_StreamStateChanged);
streamReceiver.AttachAudioPlayer(audioPlayer);
streamReceiver.Play(remoteparty);

At this point, you will only have to implement some EventHandler methods with basic functionality and your webphone client will be perfect.

Step 5. How to use your webphone?

In order to be able to use your webphone, you will need to start your server (you can do this from Visual Studio if you like) and you will need to have two computers with the access to the server application.

You will need to run the webphone client on both computers. Make sure that the server address has been set properly in the clients. The sample clients automatically connect to the server and you will start the call when there are two available (not busy) clients connected. This is the time when the Call button will be enabled.

You can start a client from Visual Studio by running it or you can use the web browser where you will need to add the html page on which your client is set.

When there are two clients connected to the server, you will be able to start a call between them and if you have done all the programming steps properly, your webphones will communicate properly.

Further development possibilities