Simplicity and Utility, or, Why SOAP Lost

tldr: Simplicity and utility trump large corporate backing.

In the middle of 2000, SOAP - an XML-based messaging protocol - started to rise in popularity as a communication protocol for distributed computing. By 2004, there were dozens of SOAP implementations in as many languages, including all of the most popular ones, such as Java, C/C++, .NET, Perl, and PHP. The term Web service became popular, describing a new model of interaction: APIs over the Web.

I was on the team that shipped various SOAP and Web services products at Microsoft. This included various product management tasks, designing features, working across companies on interoperability testing, chairing the first WS-I committee, etc. I helped build and popularize SOAP. And I watched as it lost.

Ten years later SOAP is gone. Very few, if any, developers, are creating new SOAP-based Web services. But Web services, APIs over HTTP, are more popular than ever. SOAP was popular for a while, but it lacked longevity.

 What happened?

SOAP was part of an ever expanding set of protocols that ended up getting called WS-*. There was WSDL, WS-Security, WS-SecureConversation, WS-ReliableMessaging, WS-Discovery, and so on. Each of these protocols worked with SOAP to build an entire messaging and distributed computing infrastructure. And, for many people, this looked like overkill. Sending some XML over HTTP was simple enough. But each of these protocols added complexity to that simple idea. The motivation for WS-* was sound, but it was getting marketed far beyond its usefulness.

Microsoft, in particular, was building a brand-new distributed computing framework called the Windows Communication Foundation (WCF), which they wanted to become the replacement for DCOM, MSMQ, and all of the other remoting APIs they previously offered. So, they needed a protocol that could handle not just plain RPC, but all of the distributed work you could do with the Windows platform.

In the absence of an alternative, maybe the tooling required to build WS-* APIs would have gained more traction. But Microsoft was the only major framework developer to build those tools. Simultaneously, two things started happening:

Developers needed a way to provide data to mobile clients. This was simple enough using many Web frameworks: look something up in a database, return the rows in a serialized format. Accept POSTs of the same kind of data. SOAP was doomed to failure for such simple, but very popular scenarios. It was overkill. Consider a simple Web service to return customer information.

 SOAP vs JSON/HTTP

First, using SOAP to request a specific customer record. The request will be an HTTP POST and look like this:

POST /customers HTTP/1.1
Host: www.example.org
Content-Type: application/soap+xml; charset=utf-8

<?xml version="1.0"?>
<soap:Envelope 
          xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
  <soap:Body>
    <m:GetCustomer 
          xmlns:m="http://www.example.org/customers">
      <m:CustomerId>43456</m:CustomerId>
    </m:GetCustomer>
  </soap:Body>
</soap:Envelope>

After looking up the customer, the response will look like this:

HTTP/1.1 200 OK
Content-Type: application/soap+xml; charset=utf-8

<?xml version='1.0' ?>
<env:Envelope 
        xmlns:env="http://www.w3.org/2003/05/soap-envelope" >
 <env:Body>
    <m:GetCustomerResponse 
          xmlns:m="http://www.example.org/customers">
       <m:Customer>Foobar Quux, inc</m:Customer>
    </m:GetCustomerResponse>
 </env:Body>
</env:Envelope>

Now, contrast that to using JSON over HTTP, with a slightly more RESTful orientation. The request for the customer record is an HTTP GET:

GET /customers/43456 HTTP/1.1
Host: www.example.org

And the response:

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{‘Customer’: ‘Foobar Quux, inc’}

Both of these calls are equivalent. The SOAP version sends a lot more data in both the request and response. And, because it uses XML, it’s much harder to read with your eyes. Even worse, because it uses POSTs both ways, it can’t be quickly tested in the browser.

The JSON over HTTP version is limited in utility compared to the SOAP version. Securing the request and response inside the message itself, for instance, is possible with SOAP in an extensible manner. But this isn’t a feature most developers need. The SOAP version would work over a non-http transport. But that also isn’t a feature most developers need.

 What We Can Learn From SOAP’s Failure

There are three lessons I learned from SOAP’s failure:


Thanks to David Jarvis, Bryan Costanich, Ian Geoghegan, and Rohit Rajan for their reviews.

 
562
Kudos
 
562
Kudos

Now read this

Estimate Like a Boss

It was my great luck to work with two amazing engineers at Microsoft: Erik Christensen and Hervey Wilson. One of the most important lessons I learned from them was how to estimate. I’ve modified their methods to suit my needs, so don’t... Continue →