Understanding BizTalk Messaging Services
Solutions in this chapter:
· BizTalk Messaging Services
· Document Definitions
· Distribution Lists
· Messaging Services Object Model
· Solutions Fast Track
· Frequently Asked Questions
If you have ever been part of a project in which your product was responsible for trading information between multiple systems, in multiple locations, between multiple recipients, you will appreciate the robust and feature-rich capabilities found within BizTalk Server. The Messaging Services of BizTalk Server provide administrators and developers alike the ability to configure and manage document exchange between different operating systems, different types of networks, and even different languages. How, you might ask? Microsoft has made a strong effort with their latest .NET servers and framework to support open architecture and to adhere to industry standards. BizTalk Server is no exception to this rule, and provides support for the latest XML-based technologies and operating on industry-aware and publicly available standards such SOAP and the BizTalk 2.0 Specification.
The chapter talks about the various components that constitute the Messaging Services of BizTalk Server 2000. We will discuss items such as specifications and build open the knowledge of channels, ports, etc. introduced in Chapter 3, “Testing the Installation” Upon completion of this chapter, you will have a solid foundation of the main components of BizTalk Server, and have a better understanding of the new concepts that BizTalk Server 2000 brings to the .NET server family.
BizTalk Messaging Services
The BizTalk Server framework is fundamentally based on messages. In any business process that is automated by BizTalk Server, messages must be received from their source, processed, possibly transformed into another format, and then sent to their destination. While BizTalk Orchestration services are responsible for any processing of the messages that arrive, BizTalk Messaging Services manage both the process of moving messages from their source to their destination and the process of converting documents between different message formats appropriate for each transaction.
BizTalk Messaging consists of several key objects that must be both understood and properly configured in order to correctly route messages. The main objects are document definitions, document specifications, organizations, channels, messaging ports, and distribution lists. Each of these objects depends on one or more other types of object, so it’s important to understand their purpose. A brief description of these objects is given here, and more detail is provided for each later in the chapter:
· Document definitions provide information on the type of document used by a channel.
· Document specifications describe the exact structure and layout of the document.
· Organizations represent trading partners involved in exchange of business documents.
· Channels are used to receive source documents from other organizations or other internal applications, and act as “gateways” into your BizTalk Server.
· Messaging ports are used to route documents from a channel to another organization or application.
· Distribution lists combine messaging ports into a single entity, allowing one message to be distributed to multiple ports.
Even from these short descriptions, you can see that these different objects rely heavily on one another, since BizTalk Messaging cannot perform its job without interactions between these different types of objects. While it might seem initially cumbersome to have so many different entities act in combination to provide these messaging services, it is this decoupling that truly gives BizTalk Server Messaging Services its flexibility and power. Business processes are rarely etched in stone and change frequently. This loosely coupled messaging services architecture allows you to build up your messaging system from smaller parts to accomplish your business needs. When some aspect of your business process changes, you can react by simply updating only those objects that require it (modifying a port to send an invoice to a different URL, for example), rather than redesigning your entire message transport process. The details of the interactions between the different types of messaging objects will become clearer in the sections to follow.
The BizTalk Messaging objects can all be created from the BizTalk Messaging Manager interface, which can be launched from the Start menu via your BizTalk installation. The interface is rather basic and mainly serves as a launching pad for various wizards and property pages from which you can configure your messaging objects. The Messaging Manager interface is shown in Figure 4.1.
Figure 4.1 The Messaging Manager Interface
A document definition describes a specific type of document that’s involved in a business process and any tracking or selection criteria that go along with it. It could describe a class of XML document that is received by your BizTalk Server from a business partner, or it could describe a flat-file text document that represents the output of an old legacy system in your enterprise. Document definitions reinforce the flexibility of BizTalk to understand vastly different types of documents. In order to examine how a document definition is created and what kinds of information it contains, let’s create one.
1. Choose File | New document definition from the Messaging Manager interface.
2. Name the new definition Invoice, make sure document specification is checked, and specify CommonInvoice.xml from the Microsoft subfolder in the WebDAV repository on your BizTalk Server machine.
3. The new document definition property sheet is shown in Figure 4.2.
4. Click OK to save this document definition.
Figure 4.2 Creating a New Document Definition
The most important items when creating a document definition are the definition name and the document specification. The name allows different messaging channels to reference this document definition. You should choose a name that is descriptive for this type of document. The document specification provides the structure of the XML document referenced by this document definition. The difference between a document definition and its document specification is initially confusing. However, it is important to remember that a document specification provides XML structure and content information in the form of an XDR schema, and the document definition acts as a pointer to that specification. The separation of these two concepts allows multiple document definitions to reference a single document specification, reducing redundancy in your system.
Although it is not strictly required that you select a document specification for a document definition, there are few good reasons not to do so. When you select a document specification, you get the following benefits that you would not otherwise have:
· Validation of the XML document against a schema to reduce errors.
· The ability to use the Messaging Services mapping functionality to transform one document type into another.
· The ability to specify tracking information and selection criteria for the XML document.
Without the document specification, the document definition becomes little more than a named placeholder.
Considerations for Tracking and Querying Document-Specific Data
In addition to the schema validation benefits obtained by using a document specification within your document definition, you gain the ability to track specific fields in your documents to the BizTalk tracking database in SQL Server. As a messaging channel receives a document, it looks to a document definition to describe the format of the document. The document definition describes the location of key document fields, or the important data structures contained by your document. The fields you select here are known as global tracking fields, because these fields are tracked for all documents imported by all channels that reference this document definition. This feature is useful in cases where your business needs require you to have easy access to fields.
In order to better see how to accomplish this and what benefits it yields, we will now create a new document definition. This document definition will represent an invoice document that we plan to receive from a trading partner. Because the company management requires access to important information about the invoices that pass through our system, we will need to track certain pieces of information for each invoice document that arrives. This document definition will be named Tracking Invoice and reference the same “CommonInvoice.xml” document specification from before. Field tracking options are available in the Global Tracking tab of this property sheet, as seen in Figure 4.3.
Figure 4.3 Selecting Global Tracking Fields
On the Global Tracking tab, the tree view on the left is populated with schema information from the document specification referenced on the General tab. To select a particular field for tracking, perform the following steps:
1. Click the field you want to track in the tree view on the left (note that it must be a leaf node).
2. Click the button corresponding to the desired data type. (Note that you are not specifically required to use strict data typing, but if you plan to use this information in business calculations later, it is wise to do so.)
The chosen fields appear on the right, denoted by their data type and an XPath expression specifying where to find the field in the source document. In this example, we have specified two fields from the document to track: the “Date” and “Number” fields from the “InvoiceHeader” record. After this document definition is saved and subsequently referenced from one or more channels, any documents that reference this document definition will have these fields logged in the SQL Server tracking database.
Global tracking fields specified in the document definition are saved to the SQL Server “InterchangeDTA” database in the “dta_outdoc_details” table. This database can be queried directly by your applications in order to retrieve summaries or information about specific document transactions. Note that the database schema for this table restricts you to tracking only two fields for each of the real, integer, date, and string data types. You can, however, track additional fields using the “custom” data type, which stores your additional fields (name, XPath, and value) in an XML structure in the same database record in “dta_outdoc_details.” Since retrieving those values from the XML structure is more computationally intensive than reading single values from a database, you should take care to use your strongly typed values for the fields that are most frequently accessed by your applications, leaving the “custom” structure for the less common information. If you do choose to use the custom structures for commonly accessed information, you will be responsible for parsing the custom XML structure information from the database to retrieve your values of interest and then inserting it into your downstream business logic. It is far easier for both the developer and the processor to use a single, strongly typed field from the database.
In addition to directly accessing the SQL Server database, you can view query-tracking data via the BizTalk Document Tracking Web interface. Unlike the several other Windows-based applications included in the BizTalk Server product, the Document Tracking application is accessed via a Web browser, as seen in Figure 4.4. Internet Explorer 4 or later is required to access the Web-based tracking application, since ActiveX controls are used to provide much of the functionality. This is unfortunate, since by doing things this way, Microsoft has achieved the worst of both worlds. First, due to the nature of HTML-based interfaces, Web applications are inherently feature-limited compared to their desktop counterparts. Second, by including ActiveX controls in the application, they have restricted access to users of Internet Explorer on the Windows platform. If Microsoft had gone fully either way—either a truly portable Web-based application or a full-featured Windows-based desktop client—their rationale would have been easier to understand.
Figure 4.4 The BizTalk Document Tracking Application
This document tracking page can be used to form virtual queries against the tracking database by selecting the date range of the interchange, source and/or destination organizations, and document specifications for those transactions of interest. The results can be viewed on another screen by clicking Query. The Query Results screen is shown in Figure 4.5.
Figure 4.5 Document Tracking Query Results
Individual interchanges are the main row entries, while individual documents are shown by expanding the + icon on the left of the interchange rows. The data detail contains information about the document itself, including all tracked field data.
Tracking can also be enabled at the document level, in place of or in addition to tracking at the field level. Tracking at the document level is enabled from the BizTalk Server Administration application. Right-click on the BizTalk Server Group, and choose Properties. The Tracking tab allows you to control the logging at the document level, as shown in Figure 4.6.
Figure 4.6 Specifying Document-Level Tracking Options for the Server
Document specifications describe the internal structure of your document instances. In the previous section, we learned that document definitions reference document specifications in order to provide document validation and mapping capabilities to BizTalk Messaging Services. When a channel initially receives a document, its document definition is queried for (among other things) its document specification. The document specification exists as an extended XDR file, and this file is used to validate the document instance arriving at the channel. XDR stands for “XML Data-Reduced” and is a method for describing the schema (or layout) of an XML document. Use the BizTalk Editor to create new specifications and edit existing document specifications. The editor makes it easy to graphically build and visualize complicated document specifications for various instance document formats, including XML, EDIFACT, X.12, and custom positional or delimited text file formats, although there are some tricks to generating specifications based on flat-file formats.
Here are a sample flat-file format representing a simple purchase order, and some tips to using the editor to model this format:
Joe Smith 123 Main Street Somewhere, CA 92121 123-456-7890
20 56789 Apple Pie 9.99
5 23232 Box of plastic forks 2.50
In the preceding code, the individual lines are delimited by carriage returns, while the fields within each line are tab-delimited. The steps to take to model this file in the BizTalk Editor are as follows:
1. Create an XML tree in your specification to look like Figure 4.7.
Figure 4.7 Modeling a Flat-File Format in the BizTalk Editor
2. Select the FlatOrder (root) node in the tree view on the left and the Reference tab on the right, then change the Standard property to CUSTOM. Changing this property is required to parse and validate any non-XML documents.
3. Change the Default Record Delimiter value to CR (0xd).
4. Change the Default Field Delimiter value to TAB (0x9).
5. Next, select the Parse tab on the right pane, and change the Field Order property to Infix. This tells the validator to look for the delimiters between different records or fields, but not before the first field or after the last field.
6. Change the Skip Carriage Return property’s value to No.
7. Duplicate these property settings for the Buyer node and the Item node to explicitly set the validation properties for these nodes.
8. Finally, select the Item node and its Reference tab. Change the Maximum Occurrences value from 1 to *. This is necessary to allow multiple items to be ordered in one instance of a document.
The document specification is now complete, and an instance document can be validated from the Tools … Validate instance menu option. Selecting the simple order flat file (Figure 4.7), we see the internal XML representation of the file resulting from BizTalk reading this document instance and parsing and validating it against our newly created document specification.
There is a substantial benefit to separating the document definition from its document specification. The document specification provides abstract information about the types and structures of document instances, while the document definition provides specific information about the document instance. One of the benefits lies in the ability to have multiple document definitions that reference the same document specification. This might be necessary if you sometimes need to enable global tracking fields, but not always. For example, the specification contains metadata information about the document standard (i.e., XML, EDIFACT, etc.), document type, and version, in addition to the document schema information inherent in an XDR schema. Furthermore, a simple built-in versioning mechanism is provided here, since the document specification is imported into the internal configuration database whenever you create a document definition. For example, you reference a version of a specification in a document definition and use that definition to execute your business rules. Several weeks later, you update the specification to accommodate a slightly different schema based on requirements from a new business partner, and then you create another document definition to reference the revised specification. Even when the underlying original specification file changes, your original document definition remains valid. This allows you to accommodate different versions of a document specification, each of them actively participating in your business practices at any given time.
Industry awareness and acceptance is growing constantly for XML-based transfer of business documents and for the BizTalk initiative in particular. At the time of this writing, over 500 unique schemas and specifications have been deposited in public registries such as those at www.biztalk.org and www.xml.org. These documents have been published publicly to encourage adoption of standards for data exchange within particular industries, and to facilitate communication between disparate organizations within those industries. For example, various specifications exist for human resources, transportation, and banking, and are freely available for organizations within that industry to use to enable communication between them.
One of the greatest benefits of the BizTalk initiative is that the framework itself makes no demands on the types of information that can be transferred with it. Those choices are left to those best able to determine what’s best for a particular industry or organization: the industry leaders themselves. The BizTalk Framework only specifies the packaging of the business message, requiring nothing in particular from the message itself.
Source and Destination
Another powerful feature of BizTalk Server 2000 is the ability to convert one document specification into another. This functionality is provided by the BizTalk Mapper, accessible from the BizTalk Server group in the Programs folder in the Start menu. The BizTalk Mapper allows you to create a “mapping” between a source and destination document specification, thus enabling BizTalk Server to translate one format to another as required by your business. This mapping functionality is implemented internally as an XSLT style sheet, an XML-based way to facilitate transformations between different documents. Simple one-to-one transformations can be done with simple drag-and-drop operations. More complex operations (e.g., totaling the prices for all items in an invoice and applying the result to a single field in the destination format) can be modeled by using what are known as functoids by the BizTalk Mapper tool. A functoid encapsulates some bit of logic to aid in the transformation of your document formats. Functoids are represented in the resulting XSLT as VBScript functions. BizTalk Mapper ships with 65 different functoids, divided between several tabs on the Functoid palette, ranging in function from string manipulations to math functions to extracting information from a database. For any function that’s off the beaten path and is not covered by the other included functoids, there exists a “scripting” functoid that enables you to write your own custom function to perform your transformation logic. Figure 4.8 demonstrates a map of the simple order form that was used earlier to the “PaymentSpec.xml” specification that’s included with BizTalk Server.
Figure 4.8 Using the BizTalk Mapper to Map One Document Specification to Another
Each mapped field is a simple one-to-one mapping, except for the highlighted functoid mapping. The functoid shown is the “cumulative sum” functoid, and in this case, it sums the prices for each of the items in the source document and applies that value to the Total field in the destination specification.
Developing & Deploying…
BizTalk Mapper Pitfalls
To open a source and destination specification from within the BizTalk Mapper, go to File | New. You will then be prompted to select a source specification and a destination specification. If you first attempt to use File | Open to open a source and destination specification, you will be prompted with an unintuitive error message similar to the one in Figure 4.9, even though the file is a valid document specification.
Figure 4.9 BizTalk Mapper Warning
The File | Open option is used for opening an existing Mapper output file, which internally contains the source specification, the destination specification, and the XSLT to transform between the two.
The functionality of document mapping and its use within BizTalk Messaging is discussed in greater detail later in this chapter, specifically in the discussion on Channels.
The reception and routing of BizTalk messages uses organizations as source and destination endpoints. Organizations in BizTalk are high-level identifiers for the trading partners that are source and destination of business documents. There are two fundamental types of organizations in BizTalk Server: the home (or default) organization, and trading partner organizations. There can be only one home organization (representing your internal business). In contrast, there can be any number of trading partner organizations.
The home organization is a special organization. There can be only one home organization, and it cannot be deleted. The home organization represents your internal business, and it acts as a parent for any number of “applications.” A trading partner organization can represent any of the organizations with which you exchange documents, and can act as a source or destination for your documents. A home organization cannot act directly as a document source or destination; rather, the document source and destination documents must be directed at a specific application within your organization. Similarly, a trading partner organization cannot be associated with any specific applications (since you probably do not have knowledge of the specific applications that handle the incoming documents at your business partner, anyway).
Communication between Multiple Organizations
The BizTalk Framework is centered on communication between multiple organizations. With BizTalk Server 2000, you can control the routing and processing of business documents between your home organization and your trading partner organizations. There are several paths that a document can take when routed through your BizTalk Server, depending on your individual business practices:
· Remote organization source to home organization application destination A trading partner sends a document into your BizTalk Server that is intended to be received by an internal application.
· Home organization application source to remote organization An internal application sends a document into your BizTalk Server that is then routed to your trading partner.
· Home organization application to another home organization application An internal application creates a document that is imported into BizTalk Server, processed, and the output document is routed to another internal application. This is useful, for example, when your business has multiple applications that participate in a business process but whose input and output formats are not compatible. BizTalk Server, and its mapping functionality in particular, can serve to reformat the output of the source document into the specification appropriate for the destination application.
· Remote organization to remote organization In this scenario, both the source and destination are remote organizations. Your BizTalk Server receives the document and processes and/or transforms it before ultimately routing it to the destination organization.
Each organization can have one or more identifiers that serve to uniquely identify an organization on a BizTalk Server. Each organization can have more than one organization identifier, but no organization identifier can be shared across organizations. An organization identifier is made up of three components: a name, a qualifier, and a value. The name and qualifier serve to define the “type” of information the identifier defines. The qualifier is the real key for BizTalk to determine the meaning of the value field, and it is the qualifier and value fields that are included in the BizTalk envelope header of every document in transit, ready to be interpreted by the destination BizTalk Server. The name is nothing more than a friendly name to be used when designing your BizTalk applications; BizTalk Server 2000 uses only the qualifier and value fields when examining documents and routing them to the appropriate channels and ports.
By default, when organizations are created from within the BizTalk Messaging Manager, an organization identifier is created for you as shown in Figure 4.10.
Figure 4.10 Specifying Identifiers for an Organization
This default identifier’s name is “OrganizationName,” and its value is simply the name of the organization you specified when creating the organization. This organization identifier is also marked as the default identifier, meaning that if, in your application, no specific identifier is requested for document routing, this identifier will be used by default.
Channels are BizTalk Server objects that are responsible for receiving documents from the outside world. (In this context, the “outside world” could also mean an application in your enterprise that is not well integrated into your business processes.) The configuration of a channel determines several factors about whether a document is processed by the channel and where it is routed after processing. Channels act as a sink for source documents, process and/or transform them, and then route the resulting document to an associated messaging port or distribution list. Because a channel must have a messaging port or distribution list present to act as a document destination, any messaging port or list for a channel must be created and configured prior to creation of the channel itself. As such, the BizTalk Messaging Manager will not let you create a channel by itself—the menu option is grayed out unless a messaging port or distribution list is currently selected in the Messaging Manager. Creating a new channel while a messaging port is selected causes the new channel to have that messaging port configured as its document destination.
Channels can be configured to receive documents from an organization or an internal application (of the home organization). No functional difference exists between these options once a document arrives at the channel, but this configuration option affects which documents are routed into the channel.
Channels can also be configured as “receipt” channels. Certain BizTalk documents will require that the recipient provide a receipt on processing the document. When an ordinary, nonreceipt channel receives such a document, it must notify a receipt channel to initiate processing of the document receipt. These channels and their associated messaging ports exist solely to process receipts generated by another channel/port combination.
Receipt channels are configured using the Channel wizard, just as any other channel. Because of their restricted purpose, receipt channels have slightly fewer options available to them during configuration than do ordinary channels. First, you can neither expect nor generate a receipt from a receipt channel itself. Moreover, the “inbound document definition” is defaulted to “BizTalk Canonical Receipt.” This is an internal document definition that is responsible for pulling the pertinent fields from the document envelope header and packaging them in a separate document. The document specification for the canonical receipt can be seen in the WebDAV repository on your BizTalk Server under the name “CanonicalReceipt.xml.”
User Defined Channels
User-defined channels are ordinary, nonreceipt channels. They are created and configured to receive messages, process and/or transform the messages, and then deliver them to an associated messaging port. Channels also have the following properties:
· Channels can be configured to track specific fields from the document. This is similar in nature to global field tracking via the document definition. When channel-level tracking is enabled, however, the tracked fields replace any of those that would otherwise have been tracked according to the document definition.
· Channels can be configured to contain a filtering expression, which enables them to further restrict the types of documents that are accepted.
· Document logging can be enabled at the channel level. Logging can be configured to store the inbound and/or outbound documents either in the native document format or in the intermediate XML format. This information is logged to the document tracking database.
· Security can be enabled by using the channel’s encryption and/or digital signature options.
Channel filtering is a way for a channel to further refine the types of instance documents that are accepted by the channel. A channel filter is represented by an XPath expression that returns a true/false value. If the expression evaluates to true for an instance document, then the instance document is accepted by the channel. If the expression evaluates to false, then the channel rejects the instance document. An example of selecting a filter is shown in Figure 4.11.
Figure 4.11 Applying a Filter to a Channel
This example channel filter uses the CommonInvoice specification, and sets a filter that accepts only those documents where the Seller’s contact information has the telephone number “(800)555-1212.” All other documents will be rejected.Goto Page 2 >>
|Contact Us | E-mail Us | Site Guide | About PerfectXML | Advertise||Privacy||