perfectxml.com
 Basic Search  Advanced Search   
Topics Resources Free Library Software XML News About Us
You are here: homeInfo BankArticles ╗ Making Sense of DIME Sunday, 11 November 2007
Making Sense of Direct Internet Message Encapsulation (DIME)
Matt Long (mlong@phalanxsys.com), Phalanx Systems, LLC

October 30, 2002
Phalanx Systems, LLC

Introduction

One of the reasons behind the success of XML is its textual nature, which makes XML highly portable and broadly deployable. SOAP (www.w3.org/TR/SOAP) uses XML as the messaging format. In other words, all SOAP messages are encoded using XML. The question arises, what if you were to send binary data, such as video, graphics, sound, along with the SOAP XML request or response payload? What if you were to send another XML document (encrypted, compressed, or as a fragment) with the SOAP messages?

The solutions include encoding the binary data using base64 or hex. However, encoding and decoding results in performance issues; in addition encoding to base64 or hex increases the payload size. The other solution is to use MIME (Multipurpose Internet Mail Extensions), which allows binary data to be sent and received in e-mail. However, MIME is not that efficient as it suffers overhead of buffering, and finding message delimiter strings, and also MIME can be complicated to implement and use.


The Solution

Direct Internet Message Encapsulation (DIME) addresses the difficulties involved in embedding binary data into XML documents. DIME is a specification submitted by Microsoft and IBM to IETF (http://search.ietf.org/internet-drafts/draft-nielsen-dime-01.txt), and it defines a lightweight, binary message format that can be used to encapsulate one or more application-defined payloads of arbitrary type and size into a single message construct. Using DIME, a Web service could combine text, image, and video in a single message.

In this article, we'll look at an example of generating a DIME message and then parsing it. The example makes use of DIME Generator and Parser C# classes that I have written. In this article, we'll look at the usage of these classes in a sample application. In my next article, I'll talk about the classes themselves. But first, let's review some DIME concepts.


DIME Overview

DIME is a lightweight binary message format for sending and receiving messages. The DIME specification enables one or more binary formatted payloads to be placed within a single DIME message. A DIME message is comprised of one of more DIME records which can encapsulate these binary formatted payloads. Figure 1 illustrates this structure.


Figure 1: DIME Message

DIME provides a simple solution by representing both the SOAP message and binary references within a single binary formatted message. Figure 2 illustrates the composition of a DIME message containing a SOAP message referencing attached binary data.


Figure 2: SOAP using DIME

DIME messages are comprised of individual DIME records. Figure 3 depicts the structure of a DIME record:


Figure 3: DIME Record Details

The following table describes the fields in the DIME Record:

Field Name Description
VERSION The version mask is a 5 bit integer that must be set to the value of 1, i.e., binary 00001. This is required for all records and as of this writing no other version is valid and a DIME parser must reject a message with any other version.
MB_FLAG The message begin flag is a single bit that is set to 1 for the first record of the DIME message. All subsequent records require the message begin flag to be set to 0.
ME_FLAG The message end flag is a single bit that is set to 1 for the last record of the DIME message. All previous records require the message end flag to be set to 0.
CF_FLAG The chunked flag is a single bit that is set to 1 for the first record referencing chunked data as well as all following records that include the chunked data. However, the last record containing the chunked data the chunked flag must be set to 0. Therefore, chunk flags of 1 must be associated with beginning record of contained chunked data are transmitted as ChunkedDataRecord(i) ...ChunkedDataRecord(n-1), with the last ChunkedDataRecord(n) having the flag set to 0.
TYPE_T The type mask is a 4 bit integer that indicates the format type of the TYPE field. The following table outlines the possible values for the TYPE_T field.

Name Value Description
UNCHANGED 0x00 The value 0x00 is exclusively used by all records that are chunked beginning with 2nd chunked record and used through the last chunked record. When it is used the TYPE_LENGTH is required to be 0.
MEDIA_TYPE 0x01 The value 0x01 indicates the TYPE field contains a value consistent with the BNF construct defined by RFC 2616
ABSOLUTE_URI 0x02 The value 0x02 indicates the TYPE field contains a value consistent with the BNF construct defined by RFC 2396
UNKNOWN 0x03 The value 0x03 should indicate that the type of the DATA field is unknown. The TYPE_LENGTH field is required to be 0. The DIME specification recommends that a DIME parser receiving an 'unknown' payload store the data, but not process that data.
NONE 0x04 The value 0x04 indicates that no TYPE or DATA field is used. The values of the TYPE_LENGTH and DATA_LENGTH fields are required to be 0. This value can be used whenever an empty or terminating record is used.
RESRVD The reserved mask is a 4 bit integer and is required to be 0, i.e., binary 0000.
OPTIONS_LENGTH The options length field is a 16 bit integer that indicates the length of the OPTIONS field in terms of octets, excluding any padding. Four most purposes this field will always to be 0.
ID_LENGTH The id length field is a 16 bit integer that indicates the length of the ID field in terms of octets, excluding any padding.
TYPE_LENGTH The type length field is a 16 bit integer that indicates the length of the TYPE field in terms of octets, excluding any padding.
DATA_LENGTH The data length field is a 32 bit integer that indicates the length of the DATA field in terms of octets, exluding any padding.
OPTIONS The OPTIONS field is not used for most purposes. See DIME specification for use.
ID The ID field is used to distinguish the id of the record. The id field must a be multiple of 4 octets with padding a maximum of 3 octets.
TYPE The TYPE field is used to distinguish the type of data. The type field must be a multiple of 4 octets with padding a maximum of 3 octets.
DATA The DATA field is used to encapsulate the data of the record. The data field must be a multiple of 4 octets with padding a maximum of 3 octets.


See the resources section for more information on DIME.


Sample Application

As mentioned earlier, in this article we'll look at usage of the DIME generator and parser classes that I have written. This sample C# application first uses the DIME generator class to create a DIME message containing SOAP message and a binary image data. Next, it parses this generated DIME records to get back the SOAP message and the image data using the parser class.

Download the source code files for this article, which contains this sample application along with the DIME generator and parser classes.

There are main three DIME classes:

  • DIME Generator class, which is designed to generate a DIME message from a collection of DimeRecords that are added. DimeRecords must be created and added in the order that should appear in the DIME message. The AddDimeRecord(DimeRecord dimeRecord) method is used to add the dime records in the order they should be created within the DIME message. The GetStream() method initiates the creation of the DIME messages by iterating the DimeRecordCollection and creating the DIME message. The GetStream() method return a System.Stream that represents the DIME message created.

  • DIME Parser class, which parses a DIME message as a System.Stream and can be used to return the the records of the DIME message through the DimeRecordsCollection.

  • DimeRecord Class, which is used to serialize and deserialize DIME messages.

Both, DimeGenerator and DimeParser make use of a another class DimeRecordCollection; and hence you'll find total 4 .cs C# class files in the source code download for this article. The code download zip file also contains a sample SOAP message XML file and a image .gif file.
  • Create a new C# console application.

  • Copy the DIME generator and parser C# classes (four .cs files) along with the sample SOAP message XML file and image .gif file in the folder containing this newly created C# console application files.

  • Click on Project | Add Existing Item (Shift+Alt+A) and add the DIME generator and parser classes (DimeRecord.cs, DimeRecordCollection.cs, DimeGenerator.cs, and DimeParser.cs) to the project.

  • Add the following lines at the top of the main file:
    using System.IO;
    using Dime;
  • Update the Main function with the following code: (Remember to update the path containing the SOAP XML message file and the image file, i.e localPath variable).
    [STAThread]
    static void Main(string[] args)
    {
    	/* local path of soap message (logoTransfer.xml) and image (Banner.gif) */ 
    	string localPath = @"C:\temp\ConsoleApplication2"; 
    	string pathSoap = localPath + @"\logoTransfer.xml"; 
    	string pathImage = localPath + @"\Banner.gif"; 
    
    	//***** begin first record SOAP message ***** 
    	//get the soap message and read into byte array 
    	FileStream soapFileStream = new FileStream(pathSoap,System.IO.FileMode.Open); 
    	Stream soapStream = (Stream) soapFileStream; 
    	byte[] soapBuffer = new Byte[soapStream.Length]; 
    	soapStream.Read(soapBuffer,0,(int)soapStream.Length); 
    
    	//create the DIME record for the SOAP message 
    	DimeRecord soapRecord = new DimeRecord(); 
    	soapRecord.DataType = DimeRecord.DataFormatType.Media_Type;
    	soapRecord.DimeType = "text/xml"; 
    	soapRecord.DimeData = soapBuffer;
    	//***** end first record ***** 
    
    	//***** begin last record Image Reference ****
    	//get the image and read into byte array 
    	FileStream imageFileStream = new FileStream(pathImage, System.IO.FileMode.Open); 
    	Stream imageStream = (Stream) imageFileStream; 
    	byte[] imageBuffer = new Byte[imageStream.Length]; 
    	imageStream.Read(imageBuffer,0,(int)imageStream.Length); 
    
    	//create the DIME record for the Image data 
    	DimeRecord imageRecord = new DimeRecord(); 
    	imageRecord.DataType = DimeRecord.DataFormatType.Media_Type; 
    	imageRecord.DimeID = "MyLogo"; 
    	imageRecord.DimeData = imageBuffer; 
    	//***** end last record ***** 
    
    	//instance the DimeGenerator 
    	DimeGenerator dimeGenerator = new DimeGenerator(); 
    	//add the records (in the order they appear in the DIME message) 
    	dimeGenerator.AddDimeRecord(soapRecord); 
    	dimeGenerator.AddDimeRecord(imageRecord); 
    
    	//**** return the DIME message as a stream **** 
    	Stream wireStream = dimeGenerator.GetStream();
    
    	//output the DIME Records (see parsing a DIME message below)
    	ParseMessage(wireStream);
    }
  • In the Main function above, we generate a DIME message and towards the end, call a method called as ParseMessage that parses the records back to get the SOAP message and the image file. Add the following ParseMessage method code in the console application below the Main function:
    private static void ParseMessage(Stream s) 
    {
    	//parse the message and output the soap message and image 
    	//compare the output and input files; they are the same 
    	int recordCounter = 0; 
    
    	DimeParser dimeParser = new DimeParser(s); 
    			
    	foreach(DimeRecord record in dimeParser.Records)
    	{
    		recordCounter ++; 
    		if(recordCounter == 1)
    		{
    			//SOAP message 
    			string pathSoapOut = @"c:\soapOutput.xml";    //output file for soap message 
    			FileStream fsSoapOut = new FileStream(pathSoapOut,System.IO.FileMode.Create); 
    			fsSoapOut.Write(record.DimeData,0,(int) record.DataLength);
    		} 
    		else 
    		{
    			//write the image 
    			string pathImageOut = @"c:\imageOutput.gif"; 
    			
    			//output file for image 
    			FileStream fs = new FileStream(pathImageOut,System.IO.FileMode.Create); 
    			fs.Write(record.DimeData,0,(int) record.DataLength); 
    		}
    	} 
    } 

    The ParseMessage method uses the Stream returned by DIME generator classes to get back the SOAP XML message and the image data; and saves that into c:\soapOutput.xml and c:\imageOutput.gif. Review these two files once you run the code to make sure they are exactly same as the files used to create the DIME records.

Summary

In this article, we looked at Direct Internet Message Encapsulation (DIME) specification, which defines a lightweight, binary message format that can be used to embed binary data into XML documents. We looked at a sample C# console application that used the DIME generator and parser classes to first create a DIME records containing SOAP XML message followed by the image binary data; and then parsed the records back using DIME parsing classes to get back the SOAP XML message and the image data.


Resources
About the Author:

W. Matthew Long is a rarity in the technology field, having worked as an engineer and with Wall Street investments firms, as well as being the founder of Phalanx Systems, LLC (1997) a technology consulting firm specializing in emerging Internet technologies. Mr. Long was one of the first to implement both the SOAP and WSDL specifications and was directly involved in SOAP interoperability testing from its inception. He is a regular and charter attendee at SOAP Interoperability Testing Forums. Credits include recognition by Microsoft at COMDEX 2000 for early SOAP implementations as well as being the first to implement at SOAP tool kit in Visual Basic 6.0. He has been a technical reviewer on "Programming Web Services with SOAP" and was one of the first to achieve IBM certification in XML and related technologies. His credits include technology consulting and implementation at the architectural and developmental level, and he is also well versed in business processes.

  Contact Us | E-mail Us | Site Guide | About PerfectXML | Advertise ©2004 perfectxml.com. All rights reserved. | Privacy