George Weihs is founder and CTO of RatchetSoft, a software company delivering tools and technologies designed to maximize the ROI of a service oriented architecture. He can be reached via email at gweihs@ratchetsoft.com.

Can web service adoption usher-in a collaborative development process?

XML web service technologies are rendering both platform and programming language arbitrary. Programmers are now able to build these services on their platform of choice using their favorite languages and tools. While standards are by no means complete and rival standards often confuse and demoralize the developer community, the direction of the industry is clear. Service oriented architectures (SOAs) will grow in popularity and become the de-facto way of exposing business logic.

Yet the software development process itself has seen relatively little change brought on by this new paradigm. The software development process still consists of a centralized development group that designs, develops and deploys a software product into the user ecosystem (users and IT support). Once in, the software is immutable and used "as is". The only hope of augmenting its functionality is through "feature requests" that, like messages in a bottle, often go unanswered. In fact, anyone who has ever worked for a software vendor can tell you that the vast majority of feature requests (70-80%) are never implemented because they don't have broad utility. Often, make-or-break features for one customer are unique to their use-case and of little or no value to other customers. Hence, this customer's feature request goes in - what has been affectionately called - the "Keep Dreaming" pile. This is an inevitability of a centralized software project serving multiple user constituencies for it is bound to a "one size fits all" development model.

Open source came along, and offered a ray of hope. With an open source application, the user ecosystem has the source code and can add features to the application to meet their special needs. Although a thrilling prospect, this presented the following limitations. First, all extensions must be made in the source programming language. Second, a basic understanding of the product's implementation (possibly hundreds of thousands of lines of code) is essential to changing and recompiling the source code. This one-two combination has proved a knockout blow and today the vast majority of open-source software is used "as is". So, we are back to the monolithic software model again, right? Maybe not!

There is a movement afoot to change the software development process, one that will allow end-users to actively participate through post-compile customization. This customization occurs without the need to change one line of source code. Instead, it's facilitated through the introduction of a new abstraction layer… one that when embedded in an application, translates its user interface into a service, that can be mapped to an organization's service oriented architecture. I refer to this new integration model as "Service Oriented Mass Customization" (SOMC) and it represents a dramatic change to the software development process.

SOMC, like other integration models, starts with the idea that application boundaries and information boundaries may not be the same thing. Data from applications A, B and C may collectively be required to provide the user with the "whole story". Portals are a great example of this need in action. Portals provide a snapshot of the whole story, spanning application domains and collecting all the data in one place. The problem is that it's often the wrong place. Portals aggregate data from applications A, B and C and make it available in application D (the portal itself). Wouldn't it be preferable to integrate data from application B and C directly into the user interface of application A? That's what SOMC sets out to accomplish.

SOMC allows each user ecosystem to integrate a compliant application with its unique environment. This environment includes data and business logic from various systems and vendors, all harmonized and presented as web service methods that comprise the service oriented architecture. SOMC actions are maps that visually define a simple dataflow – bits moving from onscreen fields to web services and back again. They perform simple actions like data validation and lookup to complex transactions involving other systems like ERP or CRM.

Unlike other integration models, SOMC is user driven. The end-user defines and runs SOMC actions – not administrators or automated schedulers (cron jobs). By maneuvering from screen to screen within an application, the user acts as a "record selector". They are, in effect, selecting both a schema (translated from the screen elements themselves) and a specific record (data contained within those screen elements). The user makes all the decisions about which action is run and when. They guide execution via runtime prompts for input, validation and confirmation.

Before delving into the technical details, let me first describe a use-case. Imagine a fictitious application named FieldTek. Created by an independent software vendor and distributed to customers in various industries, this application tracks field service technicians, support request and all the activities involved in managing and coordinating field service. Now imagine this application supports SOMC as a means of extending is base functionality. Let's look at the needs of two customers.

Customer A uses FieldTek to manage support for copy machines. Before SOMC, when a request came in for service, the dispatcher would look up part information and availability in their inventory management system and then rekey that information into FieldTek. But with an SOMC enabled FieldTek, the customer can "wire-up" a connection from the screen fields to an internal web service that returns product and inventory data. Now the dispatcher can simply click a button right inside FieldTek that will fetch inventory data right into the appropriate fields within the FieldTek screen. The elimination of data entry saves time and money for customer A – making it a big win!

Customer B uses FieldTek to manage support for Jet Engines. Both the FAA and the airlines need to receive this maintenance information…in near real-time. With an SOMC compliant FieldTek, the user can attach a button to an existing page (or form) and define a new behavior that reads screen information and sends it to web services exposed by the FAA and airlines. Streamlining their business process creates new efficiencies for customer B.

In both cases, the features requested were unique to that customer. It's unlikely that the vendor would assign the resources necessary to address these issues in a specific manner. Instead, the vendor can address them and many more by implementing SOMC as an abstraction layer to bridge the gap between user interface and the user's SOA.

Now let's look briefly at some implementation details. To implement SOMC in an application requires two main components: a "Reflective User Interface" and a "Service-Mapper". Let's take a minute to examine each of these components and the nature of their interaction.

Co-opting the term reflection from the object-oriented lexicon, an application that supports a "Reflective User Interface" (RUI) is one that can be queried to provide information about its current state such as onscreen fields, their data types, their current values and their read/write state. Simple UI elements such as textbox, listbox and checkbox might translate into XSD simple types such as (respectively) string, string array and boolean. More complex UI elements such as grids and tables could translate into XSD complex type arrays that represent an amalgam of simple types. The RUI is embedded in an application and its role is to translate onscreen fields to a well defined dataset and make it available as a request/response service to authorized requestors like the Service-Mapper.

The Service Mapper is a standalone application, independent of the host application. Its role is to extract data from the application's RUI and, through interaction with the user, "map" it to a web service. Then, after service invocation, the resulting data can be pasted back into the RUI. Sounds pretty simple? That's by design. The Service-Mapper is merely connective tissue with hardcore business logic encapsulated in the web service.

SOMC asks us to stop thinking of an application as an isolated island lacking a semantic bridge to the rest of the user's world. Instead allow users to build bridges by explicitly mapping elements from your application domain to others available in their burgeoning SOA. Once these bridges are built, users can take your application to strange and exotic places. They can add arcane vertical market extensions that prove invaluable to them, while remaining incomprehensible to you…and that's fine. In fact, that's the point! For SOMC is about acknowledging that one size does not fit all, and that different user constituencies may have different needs that fall outside a rigidly-defined application domain. By implementing an RUI and supporting SOMC in your application, you transform users into collaborators – building the "last mile" of the application and rescuing their productivity from the "Keep Dreaming" pile.