API Maturity in Cloud Services

I think one point people often overlook when discussing common cloud API's is the overall maturity of API's for different types of cloud services.  Trying to achieve commonality on API's that have not yet reached maturity can be a frustrating and time-consuming effort.  So, how stable are the API's for common cloud computing services?  This article focuses on a couple of different measures of API maturity to try to answer that question.

API Churn

One measure of API maturity is churn.  In other words, how much is the API changing.  Mature API's should show relatively small amounts of churn while immature, evolving API's will show increased levels of churn.  The chart below shows one measurement of churn, namely the growth in the API.  For HTTP-based API's, this can easily be measured by the number of different types of requests supported by the API.

This chart is showing us the number of API requests supported by each of the services at launch compared to the number of API requests currently supported by the services.  The EC2 API has grown from 14 requests at launch (08/24/2006) compared to 71 requests today.  This includes requests for the CloudWatch, AutoScaling and Elastic Load Balancing services which are essentially part of EC2.  Even if you exclude those services, though, the total is now 47 API requests for EC2 today, a 3X growth.  In comparison, S3 went from 8 API requests to 13 and SQS actually reduced their API requests from 12 to 11.

During this same time period, EC2 has published 17 API version while SQS has published only 4 and S3 is still running the same API version published at launch, even though some new capabilities have been added.

API Consistency

Another way to measure maturity is to compare similar API's offered by different vendors.  Mature API's should show a great deal of similarity among the different API's while immature API's will tend to show many differences.  The reasoning behind this is that a mature API should be based on a well-defined set of abstractions and therefore the type and number of API calls dealing with those abstractions should be similar across vendors.  In an immature API, the abstractions have not yet been fully fleshed out and you will see more variation among similar API's from different vendors.

First, let's compare Amazon's Simple Queue Service with Microsoft's Azure Queue Service:

This graph is comparing the number of API requests that are common between the different vendors (shown in blue) versus the number of API requests that are unique in a particular vendor's API (shown in red).  An API request is considered common if both API's have similar requests with similar actions and side effects.  Large blue bars and small red bars indicate more consistent API's whereas large red bars and small blue bars indicate less inconsistent API's.  The graph shows a fairly high degree of consistency between the two different queue service API's.

Now let's compare Amazon's Simple Storage Service with Rackspace's CloudFiles and Microsoft's Azure Blob Service:

Again, we see a fairly high degree of consistency across all three API's.  The major difference is related to the Azure Blob service's handling of large Blob's.  It places a limit of 64 MB on individual pages within a Blob and allows multiple pages to be associated with a particular page.  The other services have no equivalent abstraction, hence the differences.

Now let's compare Amazon's EC2 service with Rackspace's CloudServers:

Here we see the opposite of the graphs for the queue and storage services.  The large red bars indicate a large number of API requests that are unique to a particular vendor's API and a relatively small number of API requests that are common across both vendors.  In fact, aside from the basic operations of listing instances, starting/stopping instances, and listing/creating and deleting available images, it was difficult to even compare the two API's.  The handling of IP addresses, networking, firewalls, security credentials, block storage, etc. were unique to each vendor.

So, does that mean that common cloud API's are impossible?  Of course not.  However, I do believe that achieving meaningful levels of functionality across multiple vendors API's (especially around servers) is building on a shifting and incomplete set of abstractions at this point.  Solidifying those abstractions is key to achieving useful interoperability.