Application–to–application communication, whether over the Internet or over an Intranet, has always kept developers awake at night. Most solutions tend to be platform specific that don't scale very well, and often involve many round-trip requests from the client to the server. Other solutions, such as those involving DCOM, CORBA, etc. encounter problems communicating through firewalls. The Simple Object Access Protocol (SOAP) alleviates many of these problems. Over the course of three articles we'll examine SOAP (version 1.1) and will demonstrate how we can use JavaScript as part of our inter-application communications solution. The complete SOAP specification is available from the Microsoft web site:

http://msdn.microsoft.com/workshop/XML/general/soapspec.asp?WROXEMPTOKEN=177840ZeYjYc2L4HSCfVkq6Rxi

In order to demonstrate SOAP, I will use Internet Explorer 5 and the Microsoft XML Parser ( MSXML.dll ) – specifically I'll demonstrate a remote client in a web page executing methods on a server-based application. The full example is available for download at the end of this article. We'll also see how to transmit data between the remote application and the server using HTTP. Whether the client and the server are on the same machine, on an Intranet, or on the Internet makes no difference. More information about the Microsoft XML parser can be found here: http://msdn.microsoft.com/downloads/webtechnology/xml/msxml.asp?WROXEMPTOKEN=177840ZeYjYc2L4HSCfVkq6Rxi

However, we are not committed to using IE5 – we could equally well write a stand-alone executable (using your favorite programming language/tool) that uses SOAP to communicate with a web server (located on an Intranet or even the Internet).

Over the course of these articles, I'll be working through an example application that models a likely use of SOAP: a human resource system. The workings of the system will be almost trivial, but they demonstrate the following concepts:

Retrieving XML data from a [Access] database

Using SOAP to query the database based on some criteria

Parsing the SOAP responses (XML) into JavaScript objects, for ease of use

That's a lot for one article, so part 2 will cover:

• Parsing JavaScript objects into XML so we can use SOAP to send updates back to the server
• How SOAP represents errors and conflicts (these are known as faults )
• Using SOAP to execute a method in a server-side COM object

Ideally, you will use a production class web server for your SOAP applications, although Microsoft Personal Web Server/Services (PWS) will suffice for the demonstration code I will present (downloadable at the end of the article).

What is SOAP?

SOAP is the collective work of Microsoft, DevelopMentor, IBM, Lotus Development Corp., UserLand Software, Inc. – recently Sun Microsystems have also joined the lineup. This impressive line-up has seen previously sworn enemies come together to cooperate on the SOAP specification.

SOAP provides a consistent, structured means of transporting data and method calls between potentially distributed applications. I say potentially, because it is perfectly possible to use SOAP on a local Intranet. We could even use SOAP on a local machine using PWS – although I do not recommend creating production systems that rely on PWS.

SOAP is a text-based protocol, so it does not impose any platform dependencies. It's perfectly possible for the "client" to be a Linux application and the "server" a process running under Internet Information Server. Equally, how the server deals with the client's request is unimportant. The server may choose to use a local executable, a COM object, some VBScript or JavaScript embodied within an Active Server Page.

Platform independence is promoted further by the use of eXtensible Markup Language (XML) as the messaging format – the XML recommendation can be found at the World Wide Web Consortium (W3C) Web site: http://www.w3.org/TR/1998/REC-xml-19980210?WROXEMPTOKEN=177840ZeYjYc2L4HSCfVkq6Rxi

SOAP Message Structure

We already know that SOAP messages are encoded using XML, however the SOAP specification imposes another layer of abstraction.

SOAP messages that originate from a client application (possibly a web browser), are known as SOAP Requests . After a server has received and processed the request, a SOAP Response is returned to the client. This process is presented in the diagram below:

In XML-speak a SOAP message consists of a root element called an Envelope – this is compulsory. Optionally, inside the root node we can have a Header element. Also inside the root node, is a compulsory Body element.

Typically, each of these elements is prefixed with an XML namespace. The SOAP specification expects us to use this namespace identifier: http://schemas.xmlsoap.org/soap/envelope , and a prefix of SOAP-ENV . Namespaces (and their prefixes) are useful as they allow us to immediately identify the SOAP elements in a SOAP message, i.e. we can differentiate between the protocol requirements and the method and data content.

From the client, to the server, and back again

Communication between the client and the server is by means of the Hypertext Transfer Protocol (HTTP). In theory it's possible to use other transfer mechanisms, such as sockets, or even message queuing and e-mail.

The diagram above depicts a typical message flow. The server is acting as a "listening" device – it's looking for SOAP requests. Whilst there's only one server shown here, it's perfectly possible that the listening server could forward the client request onto another server for execution. If this model is to work, it's important to mention that both the client and the server must know how parse XML. The listening server need not know how to parse XML if it merely passes the client request on to a server that does.

Example Application

To accompany this article, I have created a small application that demonstrates using SOAP to manage employee information.

The example application consists of an employee selection form:

When an employee name is selected, a SOAP request is sent to the server. The request asks the server "please send all details about employee number 123". Assuming the server is able to handle the request, it will respond with the selected employee details. At this point, the following controls are populated:

The HTML controls are created like this:

< input type="text" name="emp_no" size="8" disabled="true">
   <input type="text" name="last_name" size="24">
 <input type="text" name="first_name" size="24">
etc.

This allows us to reference assign values to the HTML controls, like so: emp_no.value = "5";

In Practice: SOAP Request

Here is a sample SOAP request:

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope">
<SOAP-ENV:Body>
<m:getEmployeeDetail xmlns:m="http://www.craigmurphy.com/hrweb">
<emp_no>5</emp_no></m:getEmployeeDetail>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

If we remove the SOAP elements, leaving us with our message, this is what's left:

<m:getEmployeeDetail xmlns:m="http://www.craigmurphy.com/hrweb">
<emp_no>5</emp_no></m:getEmployeeDetail>

The two remaining elements consist of a method name ( getEmployeeDetail) and some data that goes with the method ( emp_no – value 5) . I've used another namespace and prefix to uniquely identify the semantics of the method call, i.e. the method getEmployeeDetail and the element emp_no should be executed using semantics provided via http://www.craigmurphy.com/hrweb . It’s worth mentioning at this point that the namespace need not be a valid web address, i.e. it can be irresolvable/unreachable – the namespace is used to provide context . I’ve used my own domain because I know it is guaranteed to be unique.

The addition of a namespace for a method allows two or more methods of the same name to be included in the SOAP body. This is a perfectly feasible situation – we may wish to call two getEmployeeDetail methods, perhaps for two different organizations.

Posting a SOAP Request

Sending this request from a client to a server is achieved using the Microsoft XMLHTTPRequest object. The following JavaScript achieves this:

var httpObj = new ActiveXObject("Microsoft.XMLHTTP");
var sSOAPRequest = "";
var sSOAPAction = "http://localhost/asptoday/hrweb#getEmployeeDetail";

httpObj.Open("POST","http://localhost/asptoday/SOAPListener.asp",false,"",
              "");

httpObj.setRequestHeader("SOAPAction", sSOAPAction);
httpObj.setRequestHeader("Content-Type","text/xml");

// Build sSOAPRequest...

httpObj.Send(sSOAPRequest);

For simplicity and readability I have left out the building of sSOAPRequest – it is just a string that contains the SOAP Request shown above.

As you’ll see later, the sSOAPAction variable is used to provide the SOAP listener with some idea about the purpose of the SOAP Request, in this case hrweb#getEmployeeDetail is considered the "intent" of the request. (This doesn’t mean that you’ll need an hrweb directory under your asptoday directory!)

This fragment does nothing more than send a SOAP request (identifying the getEmployeeDetail method) to the script SOAPListener.asp . In SOAP terminology, SOAPListener.asp is known as an endpoint . Methods are executed (or invoked) against an endpoint.

I'll explain the use of httpObj.setRequestHeader() later in this article.

In Practice: SOAP Response

Following our SOAP Request through, the server might respond with:

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope">
<SOAP-ENV:Body>
<m:getEmployeeDetailResponse xmlns:m="http://www.craigmurphy.com/hrweb">
<employee emp_no="5">
<last_name>Butcher</last_name>
<first_name>Frank</first_name>
<base_office>Walford</base_office>
<car_reg>EAST 1</car_reg>
<car_model>XJS</car_model>
<car_maker>Jaguar</car_maker>
</employee>
</m:getEmployeeDetailResponse>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Removing the SOAP element, leaves us with this:

<m:getEmployeeDetailResponse xmlns:m="http://www.craigmurphy.com/hrweb">
<employee emp_no="5">
<last_name>Butcher</last_name>
<first_name>Frank</first_name>
<base_office>Walford</base_office>
<car_reg>EAST 1</car_reg>
<car_model>XJS</car_model>
<car_maker>Jaguar</car_maker>
</employee>
</m:getEmployeeDetailResponse>

A successful SOAP Response contains the response data inside an element whose name is the original method name, post-fixed with the word Response. It's worth checking that the SOAP Response you’ve received marries up with the SOAP Request that you’ve just made. If you've made multiple asynchronous (non-blocking) requests to various servers, you might not get the responses back in the same order you sent them. Similarly, you should check the method namespace to ensure that the response belongs to the namespace your original request specified – remember you may have several getEmployeeDetail methods.

How the server creates the response will require some processing to identify the method. As soon as the method has been identified, it should be executed and a response sent back to the client. Identifying the method required is handled using the JavaScript below:

var SOAPEnvelope = Server.CreateObject("Microsoft.XMLDOM");

// Load the SOAP envelope
SOAPEnvelope.async = false;
SOAPEnvelope.load (Request);
var oBody = SOAPEnvelope.selectSingleNode("//SOAP-ENV:Body");
var oMethod = oBody.firstChild;
var sMethod = oMethod.baseName;

The SOAP message arrives at the server inside the Request object. The Microsoft XML parser allows us to load the Request object directly into an instance of Microsoft.XMLDOM . Using the XMLDOM we can call the selectSingleNode method to extract the SOAP Body, and from there we can look at the first child node – which we know will be the method required. The use of baseName simply strips off the namespace prefix. The JavaScript we could use to execute method resembles:

switch (sMethod) {
   case "getEmployeeDetail": // get the details
   default: // handle unknown methods…
}

HTTP Request Headers

In the client-side example above, you probably noticed the following lines of code:

httpObj.setRequestHeader("SOAPAction", sSA);
httpObj.setRequestHeader("Content-Type","text/xml");

The latter is the MIME type. We must set this to text/xml to identify the SOAP message to the web server – at the end of the day the SOAP message is nothing more than XML.

Using SOAP over HTTP requires the addition of a specific HTTP header: SOAPAction . Early versions of the SOAP specification used a SOAPMethodName header to identify the method being called. In the SOAP v1.1 specification, this has been deprecated in favor of SOAPAction . I have even seen some articles use SOAPMessageName , just to confuse us even further!

The SOAPAction header describes the "intent" of the SOAP request. At the time of writing, this is a free–format string so deriving any consistent meaning is going to be difficult. Surprisingly, if the SOAPAction contains a URI, it doesn't have to exist (i.e. it can be irresolvable/unreachable):

SOAPAction: http://www.craigmurphy.com/hrweb#getEmployeeDetail

In the short time that the SOAP 1.1 specification has been available, most examples you will see use SOAPAction to identify the method name. Whilst this is perfectly acceptable, we might be making more work for ourselves in the future. The method names in the Body element and the "intent" of a SOAP message are two different beasts. It's perfectly possible to have more than one method specified in the Body element – this is known as boxcarring .

The need for identifying the SOAP message "intent" is to allow HTTP services (firewalls, web servers, etc.) the chance to do some processing. The web server, for example, may re-route the message to a more appropriate server or it may use the intent to execute a load–balancing algorithm. Using an HTTP header give us the added benefit that the web server can examine the "intent" without the need to parse the XML that makes up the SOAP message.

The server-side JavaScript required to extract the HTTP header is:

var smn = new String(Request.ServerVariables("HTTP_SOAPACTION")); 
var sSOAPActionMethod = smn.substring(smn.lastIndexOf("#") + 1, smn.length);

In the absence of multiple methods in the Body element, we really should ensure that the SOAPAction method matches the Body element method:

if (sSOAPActionMethod == sMethod)
   switch (sMethod) {
      case "getEmployeeDetail": // get the details
         default: // handle unknown methods
   }

If the SOAPAction doesn’t match the Body element method, the SOAP listener should raise what is known as a SOAP fault . Equally, if your application uses "more than one method in the SOAP request" (boxcarring), then you’ll need to modify method–extraction code to suit your SOAP request.

Implementing getEmployeeDetail in this example was the trivial matter of creating a JavaScript function. However, in a production environment you may like to consider using a COM object to encapsulate your methods and business logic.

Dealing with the SOAP response

The SOAP response is nothing more than XML – thus the client may analyze it using an instance of the Microsoft XML parser. Conveniently for us, the XMLHTTP component gives us access to a property – responseXML , which is an instance of an XMLDOM. This allows us to use the following (verbose) JavaScript code:

// Send a SOAP request to get the selected employee details...
g_employee=cmSOAP('getEmployeeDetail', cmSOAPrtXMLDOM, 'emp_no', sEmpNo);

var SOAPEnvelope   = g_employee;
var oBody          = SOAPEnvelope.selectSingleNode("//SOAP-ENV:Body");
var oResponse      = oBody.firstChild;

// Extract the nodes...
var node_car_reg     = oResponse.selectSingleNode("//car_reg");
var node_car_maker   = oResponse.selectSingleNode("//car_maker");
var node_car_model   = oResponse.selectSingleNode("//car_model");
var node_base_office = oResponse.selectSingleNode("//base_office");
var node_last_name   = oResponse.selectSingleNode("//last_name");
var node_first_name  = oResponse.selectSingleNode("//first_name");

// Populate the HTML controls...
car_reg.value     = node_car_reg.text;
car_maker.value   = node_car_maker.text;
car_model.value   = node_car_model.text;
base_office.value = node_base_office.text;
last_name.value   = node_last_name.text;
first_name.value  = node_first_name.text;

As part of the example application, I’ve created a wrapper function called cmSOAP . This function takes a variable number of parameters. The first two parameters are fixed – the SOAP method name and the "return type". The "return type" specifies one of three values:

// cmSOAP Constants : return types
var cmSOAPrtXML    = 1;  // return XML as a string
var cmSOAPrtXMLDOM = 2;  // return XMLDOM
var cmSOAPrtObject = 3;  // return a JavaScript object

What follows the "return type" is any number of parameter/value pairs, or more appropriately, elements and element values, e.g. "emp_no","123". You may specify any number of elements and values depending on the parameters the method you are calling requires. In our case, getEmployeeDetail only requires the emp_no to be specified.

Using a return type of cmSOAPrtXMLDOM allows us to write code that is readable, however, it is a tedious process writing oResponse.selectSingleNode() for each HTML control. Even if we compounded the node extraction and the HTML control assignment, it's still a lot of work:

first_name.value  = oResponse.selectSingleNode("first_name").text;

If our client is a web page inside IE5, a neater solution would be to convert the XML into a JavaScript object. We can achieve this using the following JavaScript:

var SOAPEnvelope = httpObj.ResponseXML;
var oBody        = SOAPEnvelope.selectSingleNode("//SOAP-ENV:Body");
var oMethod      = oBody.firstChild;
var s="";
var r = new Object();
var node,cnode;

for (i=0; i < oMethod.childNodes.length; i++) {
  node = oMethod.childNodes[i];
  s=s+'r.'+node.nodeName+' = new Object();\n';
  if (node.childNodes.length != 0) {
    for (j=0; j < node.childNodes.length; j++) {
      cnode = node.childNodes[j];
      s = s+'r.' + node.nodeName +'.'+cnode.nodeName+'="'+cnode.text+'";\n';
    }
  }
}

eval(s);
return r;

The piece of code above is a capable of converting a simple XMLDOM into a JavaScript object. It doesn't handle XML element attributes, but it could easily be expanded.

Thus, our code to deal with a SOAP response returning a JavaScript object would resemble:

// Send a SOAP request to get the selected employee details...
g_employee=cmSOAP('getEmployeeDetail', cmSOAPrtObject, 'emp_no', sEmpNo);

// Populate the HTML controls with the employee details
with (g_employee) {
      employee.emp_no   = sEmpNo;
      car_reg.value     = employee.car_reg;
      car_maker.value   = employee.car_maker;
      car_model.value   = employee.car_model;
      base_office.value = employee.base_office;
      last_name.value   = employee.last_name;
      first_name.value  = employee.first_name;
}

The benefits of using this approach are:

• Greater readability
• The eXtensible benefit of XML is not lost – conversion is dynamic
• The object can be updated and sent back to the server (with a little JavaScript to XML conversion)

As part of the example application, there is a checkbox that allows an XMLDOM to be used, or a JavaScript object. You can then choose whichever method you prefer.

Summary

In this article, we have seen how SOAP can be used at various stages of the development process. In particular, we looked at the following:

• SOAP requests (requesting data)
• SOAP responses (returning data)
• Client/Server communication using XML over HTTP
• Converting XML to JavaScript objects conversion

You have seen that SOAP can and does work, however, whilst we have seen SOAP working, it is not entirely problem-free. Larry Masinter has created a list of issues concerning the SOAP 1.1 specification – see related links. Larry raises many pertinent points, most of which will be addressed in time, but these should not stop us using SOAP – after all, we can't sit on the fence forever! I hope that you will be able to adapt some of what you have seen here for your own work. In my next article I'll explain how to send updated information back to the server. We'll also see how SOAP allows us to deal with errors – or faults in SOAP terminology.