info@voip-sip-sdk.com Tel: 00 36 52 532 731

How to implement web to web video calls using Flash camera access

Explanation

Prerequisities

Download: 02_Web2Web_video.zip

This article introduces you the programming steps you need to do when you want to improve your Flash webphone client with video and text messaging using Ozeki VoIP SIP SDK. After reading this guide you will be familiar with all the terms related to video calls and text messaging and you will be able to write your own Flash webphone client that can handle these supports.

Introduction

Webphone technology is one of the latest and most innovative ways of communication. You can communicate with somebody from inside your web browser as the webphone itself is a web site embedded program that connects you to a remote party (Figure 1).


Figure 1 - Video communication with Flash webphones

Webphones can be click-to-call all fully features softphone solutions. A click-to-call webphone can only dial one specific number. It only needs a call and an end call function and the background activities decide who will receive the call.

A webphone with fully featured softphone functionality can be used for calling more than one client, although this solution can also limit the actually available numbers to call.

The webphone solution work as client-server applications as you cannot send TCP or UDP packets directly from and to browsers. The server is usually a console application that receives client connections and data streams and transfers them to the remote party.

This guide assumes that you are familiar with the basics of webphone programming. If you have doubts about these topics, feel free to check them on the following pages:

If you do not want to write all the necessary background support that is needed for a webphone application, you will need a good SDK to use. This guide shows you how Ozeki VoIP SIP SDK supports your programming as it provides all the tools and prewritten activities that can be needed when building a webphone application.

If you use Ozeki VoIP SIP SDK that provides the background support for both server- and client-side solutions, you will only need to set some basic parameters, specify the IP addresses, port numbers, call some provided functions and voila, your webphone will work properly.

This guide assumes that you have read the Web to web voice calls using Flash article and it will use the server and client applications of that guide as a start for this example.

The server-side application of the previously mentioned example can be used without any modifications for this purpose as it was capable of all functionalities that will be required for this new application.

You will only need to improve your client-side Flash program by inserting the code lines for camera handling and by adding some new methods for testing the camera and for text message sending.

The Web to web voice calls using Flash article shows how you can create a new Flex project in Adobe Flash Builder, now you will get to know how to import an existing project into Flash Builder.

How can you import an existing Flex Project into Adobe Flash Builder?

As you want to improve the previously written webphone Flash client with the video and text message support, you need to be able to import your previous project int Adobe Flash Builder.

If you want to import an already existing Flex project, you will need to use the File->Import Flash Builder Project... menu from the menu bar as it is shown in Figure 2.


Figure 2 - You can import your Flex projects into Adobe Flash Builder by using the menu

If you clicked on the Import Flash Builder Project... menu the Import Flash Builder Project window will appear (Figure 3) where you can specify the project to be imported. You can browse your hard drive for the location of your project file or project folder to be opened. I the case of the previous example program, you will need to specify the project folder.


Figure 3 - You can specify the project file or folder to be opened

When a project is opened, you will have to be sure that you use all the right SDKs. You can check and modify the SDK settings on the Project Properties window that can be opened either from menu or with right clicking on the project name and choosing Properties from the list that appears.

If you want to open the Project Properties window from the menu, you will need to choose Project->Properties menu that is shown in Figure 4.


Figure 4 - The project properties can be changed according to your wishes

The Project Properties window (Figure 5) provides you a lot of settings that can be made for the project. You can specify the SDKs for the project by choosing the Flex Build Path label from the list on the left side of the window.

When you have chosen the Flex Build Path label, a new panel will appear on the right side of the window with two labeled tab. You will need to choose the Library path tab and you will be able to specify the Flex version and even the other third-party SDK solutions you want to use.

As for the previously written client-side webphone solution you have specified the use of Flex 4.5.1 you can see it in the list on the panel. The Ozeki MediaGateway SDK also appears in the list as you used it too in the previous project.

You can remove any formerly added SDK solution from this list by clicking on them with your mouse. In that case the Remove button will be enabled and you can remove the chosen SDK from the list.

You can add a new SDK .swc file to the project by pressing the Add SWC... button. In this case the same browsing window will appear as in case of the project creation and you can browse your file system for the required .swc file.


Figure 5 - You can define the Flex version and other third-party SDKs for your project like Ozeki VoIP SIP SDK

After you made sure you have all the settings done right and you have all the support of the Ozeki MediaGateway SDK set, you can start improving your project. The first step, as usually, will be putting the additional elements onto the GUI.

New GUI elements will be needed for video and text message support

In the previous example the webphone was a simple click-to-call solution that was used only for voice calls. Therefore it only consisted of two buttons and two labels for information display.

This webphone example you want to build now will support video calls and text message sending, therefore it needs some extra GUI elements for the support of these functions. You can keep, however, the previously defined GUI elements as they will be useful too.

Ozeki MediaGateway SDK provides a GUI element type for camera display that will handle the local and the remote camera without anything you need to worry about. This GUI element is mediaGateway:VideoDisplayEx. You will need two of this, one for the local camera and another for the remote one. These can do with a label to inform the user which display shows which video stream.

A camera tester button could also be useful if you want to check if your camera works properly. You can put it below the local camera display and you will need to write the onclick event handler function for it later.

As for the text messaging you will need some space to type the message and some other to display the message log. You will also need a Send button that will -what a surprise - send the text message to the remote client.

If you have put all these GUI elements onto your webphone application GUI your GUI layout looks like similar to the one in Figure 6.


Figure 6 - The click-to-call webphone Flash client with video and text message support should look like this

In Figure 7 you can see the program running.


Figure 7 - The running click-to-call webphone Flash client

Now, you have your impressive GUI that can hold all the necessary functionality for your new video phoning and text message sending webphone, it is time to look at the code behind it and write some code lines to make the new functionalities work.

Code 1 shows the XML code that is behind the local camera display you have put on your GUI. You can see that this GUI element is a mediaGateway:VideoDisplayEx as it was mentioned before. You can define the ID for this panel and set the height and the width property for it.

You can see that the local camera display is set on a panel that has the title "Local camera". If you take another look at Figure 6 you can see that the title of the panel appears as a label on the GUI. This means that you will not need to define a separate label object for this purpose, you can use the title attribute (or property) of the Panel element.

<s:Panel width="100%" height="100%" title="Local camera:">
        <mediaGateway:VideoDisplayEx id="dispLocal" width="100%" height="100%">
        </mediaGateway:VideoDisplayEx>
</s:Panel>

Code 1 - The XML definition of the local camera display

The same definitions and settings can go for the remote camera display as for the local camera. Here the title of the Panel will be "Remote camera:" that also appears on the GUI but in other aspects, the two panels are exactly the same (Code 2).

You can see that the panel and the display size properties are given as percentages. This means that the panels and the displays will fill the space they get from the parent container that they are put on. You can also set absolute size properties for the panels in pixels, but this solution is more comfortable as if you want to resize the whole webphone, all the displays will be resized instantly.

<s:Panel width="100%" height="100%" title="Remote camera">
        <mediaGateway:VideoDisplayEx id="dispRemote" width="100%" height="100%">
        </mediaGateway:VideoDisplayEx>
</s:Panel>

Code 2 - The XML definition of the remote camera display

The other GUI elements work the same as in the previous example. The areas where you can type and display the text messages are TextArea objects. In the case of these objects you can set if they can be edited. In this particular webphone, the message log display should not be edited and the area where you type the messages should. You need to set the editable property of the TextAreas for these purposes.

After defining all the GUI elements, it is time for the real code writing stage. First of all you will need to insert some additional lines in some functions for the camera handling.

Camera handling means some additional code lines to the existing functions

Using Flash technology means that you use the tools and device handlers the Flash environment provides. When you want to use a web cam for your webphone, you only have to ask for it. Code 3 contains the declaration of the camera in Flash code.

private var camera:Camera;

Code 3 - There is a need for a new object that represents the web cam

When you hang up a call, the video stream will also be stopped and your webphone will be ready for another call. Meanwhile you will be able to test if your camera works as stopping a call will enable the Test camera button.

You also need to remove the video streams from the video displays on your GUI. The 9th and 10th lines of Code 4 will do this for you.

protected function btnCallStop_clickHandler(event:MouseEvent):void
{
        if (callProcess)
        {
                btnTestCamera.enabled=true;
                lblInfo.text = "Call stop, ready to call.";
                connection.invokeOnConnection("CallStop");
                ReleaseStreams();
                dispRemote.attachNetStream(null);
                dispLocal.attachCamera(null);
                btnCall.enabled = true;
        }
}

Code 4 - This is the EventHandler method for the Stop Call button

When you want to start a video call, you will naturally need to do some settings about the camera and the video displays. Code 5 shows the EventHandler function for the Call button that will run when you accepted or started a video call.

If you are using your camera for a call, you will not need to test if it is working, therefore the Test camera button can be disabled (Code 5 Line 4).

When the call starts you will need to start the input devices that are the microphone and in the case the camera. You will need to initialize the camera (Line 6), attach it to the streamSender object (Line 8) and also attach it to the local camera display (Line 10).

public function OnInCall():void
{
        lblInfo.text = "Incall";
        btnTestCamera.enabled=false;
        microphone = Microphone.getMicrophone();
        camera= Camera.getCamera();
        streamSender = new MediaStreamSender(connection);
        streamSender.attachMicrophone(microphone);
        streamSender.attachCamera(camera);
        dispLocal.attachCamera(camera);
        streamSender.addEventListener(StreamStateChangedEvent.STREAM_STATE_CHANGED, streamSender_StreamStateChanged);

        streamSender.publish(clientID);
}

Code 5 - When starting a call, all the necessary tools and devices have to be initialized and started

The OnCallStop function is one of the client-side activities that are invoked from the server. In case of the video support this method also needs some additional code lines.

You will need to add exactly the same lines to this method that were added to the EventHandler of the Stop Call button. This method will do the same as the other one. The result is the same but the source is not. This method is called when the remote party ended the call while the other will run when you press the stop button, but the environment needs exactly the same settings.

You can see in Code 6 that the Test Camera button is enabled and the camera displays get detached from the video streams exactly the same way as in case of the previously mentioned method.

public function OnCallStop():void
{
        lblInfo.text = "Remoteparty finished the call.";
        btnTestCamera.enabled=true;
        btnCall.enabled = true;
        dispRemote.attachNetStream(null);
        dispLocal.attachCamera(null);
        ReleaseStreams();
}

Code 6 - This method invokes when the remote party ends the call

When the video stream starts to play, you will need to attach it to the display it belongs as it is shown in Code 7. OnPlayRemoteStream is a function that is invoked by the server when the media stream starts to play.

public function OnPlayRemoteStream(remoteparty:String):void
{
        streamReceiver = new MediaStreamReceiver(connection);
        streamReceiver.client=this;
        streamReceiver.addEventListener(StreamStateChangedEvent.STREAM_STATE_CHANGED, streamReceiver_StreamStateChanged);
        streamReceiver.play(remoteparty);
        dispRemote.attachNetStream(streamReceiver);
}

Code 7 - This method starts the media stream playing

As you have a brand new button for the camera testing, you will need to write the EventHandler for its onclick event. The camera testing as it is shown in Code 8 is a simple method. If you are already testing the camera, you can stop the testing with the button that will change its label with every press, and if you are not testing, you can start it.

If you want to start the camera test, you will need to initialize the camera and attach it to the display. When you stop the test, the camera needs to be detached from the display and the test will stop.

private var isTest:Boolean;
protected function btnTestCamera_clickHandler(event:MouseEvent):void
{
        if (isTest)
        {
                dispLocal.attachCamera(null);
                btnTestCamera.label = "Test camera";
        }
        else
        {
                if (!camera)
                        camera=Camera.getCamera();
                dispLocal.attachCamera(camera);
                btnTestCamera.label = "Test stop";
        }
        isTest = !isTest;
}

Code 8 - Testing if the camera works properly

At this point your webphone application is fully capable of sending and receiving video streams ant its GUI can display both the incoming and the outgoing video data on separate displays. Now it is time to write the code for the text messaging support.

Text messages can be sent directly between clients

The webphone client-side applications can not send media data to each other but they are able to send text messages directly as it only means the sending of string type parameters.

However the clients can send text messages to each other without the server, they still cannot connect to each other directly. This means that only two clients that are connected through a webphone server can send text messages to each other, however, the actual text message sending will not go through the server itself.

Code 9 shows the function that can be invoked by the remote client for sending text messages. It is a really simple method that only puts the received message onto the message log display TextArea.

public function OnStreamSendText(remoteParty:String, msg:String):void
{
        txtChatLog.text += StringUtil.substitute("{0}:\n{1}\n", remoteParty, msg);
}

Code 9 - The clients can send text messages to each other without the server

You can see the handler function for the onclick event of the Send button. This will get the text from the TextArea where you typed and invoke the OnStreamSendText method of the remote party to actually send the message (Line 10).

protected function btnSend_clickHandler(event:MouseEvent):void
{
        if (txtMsgInput.text=="")
                return;

        txtChatLog.text += StringUtil.substitute("{0}:\n{1}\n",  lblClientId.text, txtMsgInput.text);

        if (streamSender!=null  && streamSender.streamState==StreamState.PUBLISHING_SUCCES)
        {
                streamSender.invokeOnMediaStream("OnStreamSendText", lblClientId.text, txtMsgInput.text);
        }

        txtMsgInput.text = "";
        btnSend.enabled = false;
}

Code 10 - This method handles the onclick event of the Send button

There is only one minor function to write that handles the keyUp event of the TextArea where you want to type the text message. This must be written as the Send button should only be enabled if there is something to send.

protected function txtMsgInput_keyUpHandler(event:KeyboardEvent):void
{
        if (!txtMsgInput.text=="" && streamSender != null)
                btnSend.enabled = true;
        else
        {
                btnSend.enabled = false;
                return;
        }

        if (event.keyCode  == Keyboard.ENTER)
                btnSend_clickHandler(null);
}

Code 11 - When you type a text message it should appear on the display

At this point your webphone application is ready for video calls and text messaging. If you have followed this guide, you now understand the basics of video streaming and text message sending, and you also know that with the use of the Ozeki MediaGateway SDK webphone applications are really easy to write.

This article showed you how to improve your Ozeki VoIP SIP SDK supported Flash webphone solution with video streaming and text message sending functions. Now you are fully capable of writing your own webphone application for these purposes, and it is time to explore more of the extraordinary support Ozeki VoIP SIP SDK can provide to you.

If you have any questions or need assistance, please contact us at info@voip-sip-sdk.com

You can select a suitable Ozeki VoIP SIP SDK license for your project on licensing page

Related Pages

Operating system: Windows 8, Windows 7, Vista, 200x, XP
System memory: 512 MB+
Free disk space: 100 MB+
Development environment: Visual Studio 2012, Visual Studio 2010, Visual Studio 2008, Visual Studio 2005, Adobe Flash Builder 4.5
Programming language: C#.NET, Adobe Flash
Supported .NET framework: .NET Framework 4.5, .NET Framework 4.0, .NET Framework 3.5 SP1
Software development kit: OZEKI VoIP SIP SDK(Download)
VoIP connection: 1 SIP account

Ozeki Cookie Policy
Ozeki Informatics Ltd uses cookies to provide you the best experience on this website. The further use of the website will be considered as an agreement to the use of cookies. For more information read this website.

Cookies are enabled You are browsing the optimized version of this website. For more information read this website.