A New IT Paradigm Requires a New IT Infrastructure
Author: Joe Labbe
Joe Labbe is CEO of Primordial, an IT consultancy that helps clients create and deploy Web services to improve processes, open new markets, and extend existing IT infrastructure.
With regard to Web services today, the question is no longer whether they will be adopted, but rather, how they will be adopted, and what impact they’ll have on the organization. Here we’ll look at the impact of Web services on the corporate IT organization.
WSDM – Web Services Demand Management
As compelling new Web services come online each day, a new challenge is emerging for the enterprise IT manager: How do I manage my organization’s consumption of Web services? The increasing prevalence of Web services will require a new infrastructure support discipline called Web Services Demand Management, or WSDM (pronounced “wisdom”). Further, the emergence of WSDM will fuel the need for a new breed of IT professional called the WSRD (pronounced “wizard”), the Web Services Resource Director.
To understand the new thinking required by the advent of Web services, it’s helpful to look to an analogy: Think back to the early days of corporate Internet use, in the mid-1990’s. At that time, provisioning ISP services was often left to the discretion of those who actually required Internet access. It was a free-for-all. As time passed, the need for Internet access within the organization became more pervasive, thus requiring centralization of the provisioning function. Once it was centralized, corporate IT could manage access more reliably and cost-effectively than any individual or department could on its own. Further, centralizing access also allowed corporate IT to provide a whole host of access value-added services such as caching and portal creation.
Based on this corollary, we believe the centralization of Web services consumption at the enterprise level is inevitable.
So what is WSDM? WSDM is the methodologies, technologies, and best practices that help organizations control and manage their consumption of Web services. The need for WSDM is rapidly becoming clear as the use of Web services becomes more pervasive. While WSDM techniques can also be applied towards managing the run-time environment of published services, current WSDM principles are geared more towards consumption of Web services where demand is highly fragmented, management controls are scarce, and the cost of service failure can be severe.
Enterprise (Centralized) WSDM
The value of implementing an enterprise WSDM strategy is directly proportional to both the volume and significance of the services consumed. As an organization’s dependence upon both internal or external Web services increases to facilitate important business functions, so will the need to centralize access and standardize service handling. Organizations that rely on each individual service requestor (i.e., software program that calls a Web service) to handle such complex issues as security and authentication, service monitoring, contingency and results caching, will experience significant scaling problems. Further, as desktop productivity software products (e.g., word processors and spreadsheets) evolve to also include Web services capabilities, the effort associated with enterprise Web Services Demand Management will become more arduous.
The only way WSDM best practices can scale is through centralization. While it is true an organization can employ some WSDM best practices through policy-driven standardization (i.e., “everybody remember to do these things when you call a Web service”), this approach usually falls short due to the overwhelming number of failure points. Experience tells us it’s very difficult to regulate the behavior of developers across the enterprise without implementing a systematic compliance mechanism. This mechanism should both enforce policy yet also remove the burden of each developer having to repetitively code common service call requirements such as logging, service contingency or caching.
Once an organization adopts WSDM at the enterprise level, best practices recommend the following functions be abstracted from the “end user” calling application:
- Service monitoring and reporting
- Service contingency and switching
- Security and authentication
- Connection pooling and results caching
- Enterprise sponsored XML transformations
Service Monitoring and Reporting
Since WSDM is primarily a service infrastructure discipline, comprehensive service monitoring and performance reporting is a critical control to help safeguard service reliability. WSDM helps organizations understand:
- What services are being consumed?
- Which applications are consuming these services?
- How are these services performing with regard to past performance and/or the performance of similar services?
- Are the providers of these services living up to their service level agreements (SLAs)?
- How much money is my organization spending on the consumption of these services?
- Can we consolidate service consumption and negotiate volume discounts with providers?
- Are there new versions of these services available and do they possess important new features?
This kind of valuable tracking information would be quite difficult to ascertain if each requesting application was required to handle service monitoring on its own. Further, WSDM greatly enhances an organization’s ability to allow for real-time service monitoring either through a WSDM “cockpit” or SNMP enabled tools.
Service Contingency and Switching
Unfortunately, service failures are inevitable. In the event a service fails or an endpoint changes, it is critical that consistent service contingency and switching rules get automatically executed. WSDM ensures that service access errors are handled in an efficient and consistent matter. For example, from time-to-time, the publisher of a Web service will change the service’s virtual endpoint. Often this is due to hosting environment changes or the bad side effect of a new version launch. Regardless of the reason, SOAP best practices dictate that upon receiving a service access error, it is incumbent upon the service requestor to check the public UDDI or the organization’s private partner catalog to try and locate the service’s new endpoint. Unfortunately, leaving such contingency procedures up to the discretion of each application developer will introduce all kinds of service recovery inconsistencies. Centralizing Web service access handling and executing “self-healing” procedures is the recommended approach.
Further, WSDM also calls for service contingency and monitoring procedures to be implemented in tandem, so it’s possible to identify and fix potential service access problems before a call is made from a “hot” requestor application.
Security and Authentication
Today, most Web services calls contain login and authentication information. While this might suffice for the moment, as organizations migrate mission critical functions to Web services, this practice will prove inadequate. Enterprise WSDM principles dictate that service login and authentication information be abstracted from the end user calling application and centrally stored at a Web service proxy layer. Under this scenario, the end user calling application (residing inside the organization’s firewall) authenticates to the Web services proxy layer. Then which upon authentication, inserts the organization’s secured authentication information into the payload and passes the message on to the actual Web service. This will not only make the transaction more secure, but will also ease maintenance efforts when security modifications need to be made.
Enterprise Connection Pooling and Results Caching
While we anticipate the SOAP specification will eventually include provisions for both connection pooling and results caching, currently, there are little by way of standards-based controls to perform either function. In the meantime, WSDM employs certain techniques to bridge the specification gaps. For instance, firms that funnel all SOAP requests through a central Web services proxy layer can make more effective use of the “keep alive” SOAP action and in effect simulate connection pooling. This assumes the Web services publisher makes keep alive available and the proxy knows how to intelligently batch requests made of the same service. Response times should improve based on decreased connection setup and breakdown cycles.
Results caching is a bit more complex. Although results caching can improve response times dramatically, predictive caching can be dangerous unless the underlying service is well understood and parameterized properly. Caching stock quotes is a perfect example. During the course of the trading day, quotes for a given equity can change from second to second. Obviously, caching should not be used to store data that changes this frequently.
However, after the trading day closes, the quote for a given equity will not change until the market reopens on the following day. In this case, caching can dramatically improve response times without jeopardizing accuracy. The only other issue to consider when exploring caching strategies is whether or not your service provider allows results caching. This is especially applicable as pay-per use services become more pervasive.
Value-Added XML Transformations
One of the most valuable benefits of implementing a centralized WSDM strategy is the ability to perform global XML transformations. An example of this power was demonstrated above when we addressed the issue of security abstraction. But it doesn’t end there. Implementing this best practice allows organizations to “treat” or transform XML payloads based on enterprise processing rules. This not only removes the burden of each application developer having to remember to enforce these transformations, but also reduces the chance of error and decreases maintenance headaches.
The WSRD – Web Services Resource Director
Who should be responsible for implementing and supporting WSDM within an organization? Currently, there is no clear answer due to the fact that the skills required to perform the job effectively are not often found in just one IT professional. The skills required are in the areas of network engineering, application development, process and configuration management, and vendor relations. The need for this convergence of skills in one individual should lead to the creation of a new IT professional whose charter it is to define, implement and support WSDM initiatives across the enterprise. We call this new IT professional the “WSRD”(pronounced “wizard”). The WSRD will be responsible at a minimum for the following issues:
- Defining the WSDM architecture
- Tools selection (proxy, monitoring, load balancing, process management and queuing)
- Service monitoring and reporting
- Service contingency and switching
- Financial tracking and least cost routing
- Vendor negotiations (in tandem with business function representatives) and SLA compliance
- Enterprise WSDM compliance training for application developers
As Web services become more pervasive, loosely defined, point-to-point service access methods will give way to well-defined enterprise WSDM practices. This evolutionary advance towards standardization will be critical if Web services are to gain legitimacy.
Maximizing the value of Web services calls for a new set of management skills, tools and methodologies within the standard corporate IT structure. The sooner this new way of thinking is adopted and supported, the sooner companies will begin to reap the many benefits of this transformation in enterprise information technology.
Note: This article is published with kind permission of Primordial.
Primordial is an ebusiness consultancy that helps Global 2000 companies conceive and execute Web services to evolve existing IT infrastructure and realize greater return on assets. Through process orchestration and application syndication, Primordial helps businesses achieve operating efficiencies and generate revenue opportunities throughout the value chain. Primordial is a member of the World Wide Web Consortium® (www.w3c.org), the non-aligned body that evaluates and adopts Web standards.
A Day in the Life of the WSRD
The WSRD – or Web Services Resource Director – is an integral part of the new IT infrastructure. Think you’ve got what it takes to be a WSRD? Before you answer, take a look at the typical day-in-the-life of the corporate WSRD.
Jim Ryan was recently promoted to the position of WSRD for a mid-market electronics component manufacturer. Prior to the promotion, Jim was a Director of Systems Architecture and will need to draw equally on his experiences in application design and infrastructure support in order to perform his role effectively. The following chronicles Jim’s typical day as a WSRD.
7:45am – While on his way to the office, Jim receives an email notification on his text pager that a Web service his company relies on for assessing currency risk associated with international sales from their Web site has been down for fifteen minutes. The message was sent from the WSDM proxy server Jim installed last month. The message alerts Jim that the service is down and that a contingency rule has been invoked. The contingency rule dictates that all requests for this service be re-routed to a secondary provider until the primary service is restored. Further, the WSDM proxy server has also sent a message to the provider’s customer support Web service informing them of the outage and that the duration of this outage is breaching their negotiated SLA. Since Jim created a common interface to both the primary and secondary service last month, the switch is made without any action needed to be taken by the consumers of the service. Jim will follow up with the primary provider later in the day.
8:30am – Jim arrives at work, loads his WSDM cockpit (monitoring dashboard) and reviews a series of traffic and financial reports. These reports let him know what services were consumed, by whom, how those services performed, and what are the financial implications of those services calls. The report clearly notes the currency service outage and the SLA violation. Jim then loads the real-time traffic monitor, sees the secondary service is functioning well and puts in a call to the primary service vendor.
9:15am – Jim rushes to a meeting with the folks in logistics. They have been designing a portfolio of Web services that will grant their customers greater visibility into order status and shipments. For a period of time, these Web services will need to work in tandem with the company’s EDI program so Jim has been asked to advise on the interface design. Since Jim manages the consumption of Web services within his organization, he is best prepared to help the Web services architects design interfaces that comply with WSDM methodologies. Jim represents the customer in these meetings.
11:30am – Jim checks his email and notices he has received four requests for access to new external Web services. Jim will need to work with the requesting parties (both line of business and application development managers) to help evaluate the service vendors. Further, Jim will also get the ball rolling regarding technical contract issues, developing service profiles and processing rules so the services can be registered with the WSDM proxy server.
12:45pm – Jim is called into a meeting by the business process management team to discuss implementing a new workflow for direct marketing campaigns. They would like to streamline the process by outsourcing certain tasks to credible providers and utilize Web services as the interface. The team would like to work with Jim to find providers that can broker lists, perform address verifications, handle materials fulfillment and provide campaign feedback to internal CRM systems. Jim will ultimately help assess providers, negotiate contracts and define workflow patterns. This is the first real world opportunity for Jim to implement that BPM package he’s been playing with for the last two weeks.
2:30pm – Jim receives a notification from the WSDM cockpit that a service provider has expanded their service to include new functionality. Jim accesses a version of the new WSDL document and performs an impact analysis. The news is good. The service will be backwardly compatible and support all the old interfaces. What does catch his eye is that there are several new functions being made available for which the users have been clamoring. While no immediate action needs to be taken to continue smooth operations with the service, Jim does set up a meeting with both the business unit sponsors and lead developers responsible for applications that consume the service and discuss the viability of accessing the expanded functionality.
3:00pm - Jim continues work on converting the corporate XML vocabulary so that it complies with the latest RosettaNet standards. This has been a time consuming process but is vital to the company’s interoperability goals that they establish a standard ontology.
4:30pm – Jim meets with a Web service network vendor. Jim is looking to incorporate a Web services network into the company’s messaging infrastructure to help with security and non-repudiation. This week, he is evaluating both a physical and a virtual peer-to-peer solution provider.
6:00pm – Before he leaves for the night, Jim checks with one of his staff members to find out how they are progressing with setting up a training program for end users that are looking to consume Web services directly from productivity tools such as Word and Excel. Jim realizes that as end users consume such services from these tools, traffic will grow exponentially. So will the support headaches and Jim wants to make sure that folks understand the service provider evaluation, selection and proxy registration process. Maintaining control will be key to providing reliable service.
6:30pm – On the train ride home, Jim starts to thumb through the latest version of the SOAP specification document. He knows he needs to start implementing digital signatures and encryption so he is hoping to get some insight regarding how to proceed based on the direction of the standards. This has been a bit of a moving target and he wants to make sure that whatever he implements it’s based on the standards. He is interrupted by a message on his text pager yet again. This time the news is good. The primary currency service provider is back up and the WSDM proxy server has made the switch back flawlessly. Tomorrow, Jim will contact the provider and negotiate a discount based on additional costs he had to incur when he switched over to the secondary provider.
Back to Articles Page