There have been a number of interesting tracts written lately about that increasingly popular topic in the world of SOA and Web services: REST. In particular, the one that is circulating around social bookmarking sites and SOA blogs the most in the last month is Stefan Tilkov's excellent Addressing Doubts about REST. The article tackles the continuing skepticism that SOA practioners have had about the integration approach that has become the dominant one on the Web and is now making significant inroads in the enterprise (more on my findings on enterprise adoption of REST here soon).
Key SOA Trend: As of March 2008, leading industry analysts -- such as Anne Thomas Manes -- are concluding that "SOA is not working in most organizations."
Compellingly, Stefan goes far beyond the simple and often misleading SOAP vs. REST debates and makes a number of excellent points about the REST approach ranging from encapsulation and transaction boundaries to documentation and tool support. But what struck me most is that these largely technical concerns, while very important, still don't strike directly to the heart of what makes REST, and Web-Oriented Architecture in general, so significant to the practice of effective large scale software integration and composition. Specifically, the more I look at working examples of large scale SOA on the Web, the more I'm aware of the fundamentally different mindset and approach that are used by those that have an urgent business need to achieve deep levels of integration between many customers and trading partners.
So what are the big differences between traditional SOAP-based, top-down SOA and lightweight, bottom-up WOA? In the end, it's as much architectural and philosophical as it is technical. I'll also be clear and note that while successful large-scale SOA on the Web tends to favor REST, REST drives many of the concepts described below, rather than promoting them explicitly. In other words, REST resides at the core of Web-Oriented Architecture, which in turn describes a set of related approaches for creating a robust and bustling network ecosystem of loosely cooperating entities that typically compete for consumption via "architecture of the fittest." Here are some of the key things we've learned over the last half-decade that REST has been used widely to build WOAs:
12 Things You Should Know About REST and WOA
- REST posits an interconnected information ecosystem, not an isolated set of point Web services. REST services are (usually) XML resources that are deeply linked together using URIs (connectors in REST parlance) into a tapestry thousands and thousands of other Web resources. This is the model used by the Web itself, which uses the same model (thousand and thousands of HTML resources deeply linked with URLs). The key concept here is that REST resources can be linked with other Web resources made by the same, or entirely different, providers. If you build a REST Web service and publish it, it's highly likely that in a short while you'll be a referenced resource in another REST service. While this sort of data transparency seems in direct opposition to widely cherished beliefs in the software development community around concepts like encapsulation and separation of implementation from interface (aka information hiding), it turns out that the "side effects" of this kind of transparent information architecture are many, varied, and usually highly desirable. The Web has taught us that publicly visible deep links are enormously important to system architecture, even as important as the data itself, enabling vital scenarios like discovery, search, analytics, transparency, participation, increased consumption, high levels of scalability, and well, robust ecosystems of participating components that can openly consume (and sometimes operate) on this data. As a final note, REST services can in fact still completely separate interface and implementation while at the same time prescribing a specific set of interaction scenarios.
"...the interconnected galaxy of data itself is now the central construct that is consumed and operated upon by network components."
A focus on Design for Consumption instead of Design for Integration. While some SOA traditionalists might disagree, there is a tendency to focus excessively on the imaginary integration point, or seams, of an SOA using the traditional WS-I Basic Profile world and I've had long conversations in the SOA community about tools for schema conversion, data translation and mapping, and other complicated scenarios to make two endpoints talk effectively. In this view, both sides of the conversation must have the same exact lens on how to approach the integration process, or agree to disagree. I've called this the "tyranny of SOAP's mustUnderstand flag" and this, combined with the fact that you almost always had to have the same programming language and Web service toolkit at the exact same revisions on each side of the conversation, it results in practice in surprisingly low levels of practical interoperability. Traditional SOA is designed, by intent, to diverge and fragment both because of the design of SOAP but also the proliferation of dozens of heavyweight WS-* standards that put a very heavy consumption tax on the conversation. The XML Schemas (or XSD) used in WSDL have also turned out to be a rather poor choice for meaningful descriptions of information that pass across integration points. This is in sharp relief to the world of REST and WOA where extremely simple standards ensure that whatever programming language and toolkit is being used, as long as it can process simple HTTP and XML, can interoperate quickly and easily while referencing the services API documentation. In other words, WOA enables integration between anything that can process the Web while SOA enables integration only between the (increasingly rarified) stacks standards and protocols that a given traditional Web service endpoint supports. Thus REST posits a world of integrating entities containing an almost infinite diversity in participants that couple well and scale best because of extreme simplicity and very low barriers to consumption. In contrast, there are only a handful of SOA toolkits that have the levels of sophistication to handle the fuller vision of heavyweight SOA, and unless you're using them, you can't come to their party. Finally, another way to look at this is that REST is near the top of the tolerance continuum and thus will always be significantly more open, inclusive, and egalitarian from a consumption perspective. If maximizing opportunities for integration is your goal, the right approach for you should be clear.- REST security is egalitarian and is as secure as the Web itself. Some theorists will raise the concern that using protocols such as HTTPS to secure REST is like using a single blunt instrument to solve a delicate and sophisticated set of problems. In practice, the protocol that has successfully protected the majority of e-commerce transactions on the Web is good enough for most applications. If different or more sophisticated means are needed, you can enable them as well, but it's clear that the large SOA practitioners on the Web are not adopting standards like WS-Security. For example, Amazon's popular S3 service uses simple, straightforward HMAC-SHA1 signatures to handle the authentication of each and every request to its REST API. Balancing security with ease of consumption requires a careful tightrope walk when it comes to successful software integration and the security solutions for Web services being using on a large scale today are not the ones we expected 5 years ago.
- Service interaction directly by the client is a first class citizen in WOA. You might be reading this and thinking it's a no-brainer. But client consumption is a surprisingly slippery subject in the world of traditional SOA. For one thing, many SOA architects still refrain from thinking of the application client as a place where services are consumed directly, at least as a primary architectural concern. The client in this view can be the browser, mobile device, native application, or whatever is being used. However, the rise of rich user interfaces as well as the mashup software development model has driven the requirement for many Web services to be accessible directly from the client as a first order design concern. However, this is where traditional SOA has had significant issues. SOAP, the fundamental SOA protocol, does not have direct support in any of today's Web browsers making direct consumption problematic for even simple SOA services, and quite difficult for WS-* style services. Even worse, the latest new rich Internet application platforms such as Adobe's Flex, which are have become true software development environments in their own right , often have surprisingly poor support even for such important standards as XML. That's not to say that adapters, bridges, proxies, and other solutions can't be applied to existing SOAs to project them into the client erna. But all of these bring their own architectural tradeoffs and needless (and expensive) complexity including more layers of data mapping, translation, and run-time performance. Like other fundamental protocols such as RSS and ATOM, which are directly consumable by virtually all clients today, the very best SOAs make service consumption by the client a first class citizen and its services highly consumable in any rich Internet application, Web mashup, mobile device, or from wherever it needs to be accessed. This is key aspect of WOA's Design for Consumption, like Design for Manufacturing did for a generation of engineering processes and directly enables many important new scenarios we are looking for in software integration and composition today.
- Service contracts are simpler and suppler in a REST model. Coming from a formal software engineering background, I myself was on the fence on whether the lack of a traditional interface contract actually inhibits the high volume consumption of REST services, which as we've discussed, is supposed to be one of its great strengths. This is one area where traditional SOA appears strong on its face, using WSDL to describe and elaborate on the precise nature of the methods, structures and data types being passed back over the integration point. While the topic of the impedance between most programming languages and service contract formats is beyond the scope of this discussion, suffice to say that we've learned over time that WSDL generally encourages tools to be far too finicky about a service contract and tends to create a brittleness that doesn't need to be there. I've written about minimal surface area dependencies for Web services before and it turns out that the everyday pragmatic consumption of REST is just not hindered by lack of machine readable contracts. For a variety of reasons, this tends to encourage a dependency on just the parts of the service begin used, and not the entire service. While the formal computer science crowd will have concerns about building reliable systems on top of services that change over time and lack formalized, machine consumable contracts, the reality is that in an environment that increasingly seems to be heading for a much higher number of informal services produced by a much higher number of sources, the lack of formal contracts is increasingly a feature. In fact, contract by example is sufficient for most applications, although mature offerings meant for transactional use do tend to have a formal API description, just not always machine readable. The early industry SOA assumption that most users of Web services would only consume them when aided by sophisticated tools has not been borne out on the Web. In fact, the world of ad hoc integration via mashups has further shown this not to be an issue. However, solutions such as WADL seem to be gaining currency when you absolutely need a contract for a REST service, though most developers I know using REST are more than content to just interact with the service itself or use a wrapper library that is provided by the service creator or the community that's grown around the SOA or API itself.
Figure 1: All Web services and REST resources have a contract, implicit or explicit.
- REST strongly complements traditional SOA, if you must have it. Though increasingly, you don't have to have it. REST generally has much better consumption scenarios, is faster, more reliable, and more likely to be usable by those on the other end of the network conversation. Wrapping SOAP and other SOA-style services in REST is a workable solution, depending on what you're doing. Heavyweight service-orientation is at the bottom of the tolerance continuum and can make sense of a specific set of requirements, but chances are that REST will give you most flexibility, options, and uptake.
- REST and WOA enables and does not violate the principles of service-orientation. Thomas Erl, one of the leading SOA thinkers in the industry, has identified eight principles of service-orientation that are generally agnostic of the technology used to implement a SOA while directly supporting the reported benefits of a workable service-oriented approach including easier interoperability, high levels of reuse, more flexibility in design, and so on. These principles include abstraction, loose-coupling, service-contract, reusability, autonomy, statelessness, discoverability, and composability. All SOA implementations tend to comply with or violate these principles to a varying degree either intentionally or unintentionally depending on their requirements and other vagaries. In this way, each SOA implementation has countless accumulated design decisions built into it that embody the architects', implementors', and vendors' net assumptions for the best way to realize the services that comprise that SOA. REST and WOA bring their own unique emphasis around what important in a service landscape, but critically, these do not violate a single of the essential architectural principles of service-orientation and often enables them unique and powerful ways. I'll explore these individually as I am able in upcoming posts since the statelessness and service-contract principles are very interesting areas for many SOA implementors to understand in a REST world.
Industry PerspectiveEnterprise IT and SOA experts David Linthicum and Dana Gardner have recently weighed in on WOA on both
Infoworld and
ZDNet.
We have reached a possibly final state of deconstruction between data and function. I only say final since the Web is increasingly having the last word when it comes to the largest and most successful examples of just about any type of system you can describe and we don't see anything emerging beyond it. And Web has an intrinsic model that is exerting a network effect of its very own; if one builds something now that doesn't align closely with the grain of the Web then it will get largely sidelined until it is somehow woven into it. In other words, build a Web service that's not Web-oriented and chances are good it will stagnate. But build one that's Web-oriented and thousands of people will likely beat a path to your door (there are other success factors here of course, such as having best in class data). But there's a very big discussion lurking here with the essential idea is that we've nearly come full circle from the days of object-orientation where objects were code that was very tightly coupled to the data it operated upon. At the time, it was an architectural concept, not just for local information hiding. We moved from there to distributed stateful objects, then distributed stateless objects, then components, network services, and many other models. Services and code, however, tended to have the upper hand overall and mostly stood in front of the data or the database. But we've undergone a thorough inversion of this model because of the growth of Web architecture and the interconnected galaxy of data itself is now the central construct that is consumed and operated upon by network components (code running on servers and clients). This is a very different worldview that we have had in most of the traditional software industry, but the Web itself has essentially trumped the conversation and provided us with what appears to be the most workable model yet for the architecture of highly federated systems and composite applications. And this new lens is very Web-oriented.- REST drives WOA but WOA extends beyond REST. I realize that WOA isn't a fully accepted industry term yet, but I do favorite it to terms like resource-oriented architecture (ROA). WOA does indeed start with REST but also encompasses and intentionally extends into other, closely related models for designing and distributing composite systems and services. A good example is the rise of the Web widget as one of the newer and more interesting "component models" for enabling distribution and consumption of REST-based services. The Google Maps widget is one pre-eminent example of how WOA goes well beyond simple REST and describes a complete and integrated "package" for a WOA capability that offers an open API via Javascript which provides deep access to remote Web services. All this and it's also in a nice and clean browser-side API that even includes best-of-breed visual functionality. In this architectural worldview, you take a much broader perspective on opening up and offering services that provides delivery all the way to the "last inch" and gives developers data and functionality in a format that allows an SOA to be consumed in the simplest and lowest barrier fashion. The reward for Google has been one of the highest rates of uptake within 3rd party composite Web applications (aka mashups) of any API on the Web. The SOA model here is infused with the latest open Web-based user interface approaches to the degree that even a default presentation reference model, ready for production, is included as part of the SDK. That's as a complete, holistic, and pragmatic a view of SOA as you're likely to see since the most useful and productive consumption models in service interaction are emphasized while at the same time tightly aligning with the architectural and application model of the Web. This is what WOA at its finest can represent. Off the shelf WOA components like this are springing up and being used all over the Web (tens of thousands of them can be found on Widgetbox and the Google Gadgets directory), but are very hard to find in a traditional SOA environment.
- REST is deeply infused into the fabric of the Web today. Not only is every single hosted Web page presently in existence already a read-only REST Web service (in REST parlance, transferring the representation of the state of the page via HTTP using the GET verb), but the latest and most influential Web standards, such as the highly regarded Atom Publishing Protocol, are inherently REST-based as well. Thus the overwhelming majority of pure data Web services on the Web today are REST-based, particularly the several hundred million RSS endpoints that are currently live right now. I've heard multiple times the story of how an enterprise switched from SOAP-based services to open syndication models for example, because many more tools support simple data pulls over HTTP, never mind the other advantages we've already seen above. We are just not seeing that sort of organic uptake and pervasive adoption with traditional SOA technologies. That is not only because of the aforementioned network effect but it also takes into account the very important lessons that we've learned from the Web. And one paramount lesson, as we'll see, is not controlling the other side of the conversation, which is one the last big pieces of the WOA picture. Which is....
- REST enables an inversion of control that drives adoption and integration. This is somewhat similar to the inversion of control we see in things like dependency injection, in that the more direct control we give up over the integration process, the most integration we get because we've enabled the scenarios for it out in the "cloud" of the network. In a very similar way that the hyperlink itself -- and the URI in REST -- allows anyone external to the linked resource to connect information together, without a finger being lifted by the originating resource, the REST model allows what some call outside-in integration with the potential of almost entirely allowing integration to happen entirely external to the integrated system. While you might be thinking that surely our SOA approaches up until now have enabled this, the practice has been creating an escalating stairway of barriers to hurdle: You must process all protocols layered in the SOAP envelope to participate, you must have a contract in WSDL, there's a strong preference for information in XML, you should use the same programming language/platform as the service provider to avoid translation bugs, and so on. These and many other requirements impose a great deal of unnecessary control on both sides of the conversation but particularly on the consumption side. We seem to be learning that the very best models for integration impose as little control as possible. REST informs us that we must have a common representation of state, but it could be XML, or JSON, or images, or video. But beyond that, we are not constrained as long as the representation will fit over HTTP. And in this way, control over integration is inverted to the consumer of the service, who can engage in a thousand new scenarios not possible when all the aforementioned constraints are made. REST can set Web sites, businesses, applications, and every other silo you imagine as free as we know how to do it. As simple as possible, but no simpler and thus the network can integrate itself and we can achieve the advantages of pull instead of push, fluidity instead of impedance, a bazaar of consumption instead of a cathedral of integration.
- REST and WOA can handle systems of arbitrary complexity and size. The systems built today from Amazon's Web services and many others show that hundreds of thousands of customers can integrate effectively and operate simultaneous on the Global SOA and run their businesses using REST and WOA. This is the "my Web site is bigger than your enterprise" realization that is making enterprises look hard at what's actually working on the greater Web. REST and WOA are not just ready for prime-time, they are prime-time.
There are literally dozens of models for building services to connect systems together. However, HTTP is at the core of many of the more promising ones, including REST. Here's an overview of the most common service models for SOA today.
Conclusion
However, many readers of this article are doubtless still wondering if REST and WOA are really the end-all, be-all for service-oriented architectures. For now, we're seeing it as one of the best available options despite a great deal of work yet to figure out how to apply it fully to the world of the enterprise. Is it ideal for every single type of application and scenario? Of course not. Your mileage will vary entirely depending on your requirements and your understanding of how REST deeply informs system architecture. However, it's increasingly emerging on the short list for those integrating systems of even the very largest size and complexity as well as down to the simplest and most nimble application. We've also learned a lot about the strengths and its weaknesses of this Web services model, however, as a fundamental part of the Web (since REST is nothing more than HTTP applied to data), REST along with WOA is the model that underpins many of the largest and most successful networks (and SOAs) in history.
Finally, I'll be delivering a complimentary Webinar on this subject matter next week, on Thursday, April 17th, to present the full scope of WOA and how it can be used to drive adoption, better business outcomes, and make SOA work in the enterprise. I do hope you attend.
Posts
