Information Integration Patterns

By SD Times News Team

January 1, 2008 —

I’ve been thinking about the patterns of information integration that we see these days, as related to SOA, in products as well as in custom solutions: information- or service-oriented. I’ve also been thinking about levels that make up these stacks, from the most primitive to the most sophisticated.

Keep in mind, there are differences between information/data integration, as related to SOA, and transactional or behavior integration, as related to SOA. Information integration…is…well…information integration, or, the structured movement of information between one or more systems. Transitional/behavior integration is the abstraction of functions between two or more systems. If this is as clear as mud, you’re right. There are always overlaps and exceptions, but it’s a good idea to consider both approaches since the technology you select, as well as the solution patterns, are a bit different.

Not that these kinds of reference models provide a huge advantage, but they do provide a checklist of sorts that allows you to better understand what you may need when building your SOA, and a good framework of understanding for the enterprise.

Considering that preface, I’ve come up with nine levels of integration that seem to cover the patterns I’m seeing as enterprises are building SOA. Here they are:

• Level 0 Simple information flow and management, no transformation and routing. This just means information is flowing from one side to the other. In essence, this is a simple pipe where data flows from point A to point B.

• Level 1 Simple information transformation, but no logical operators for routing or message processing. This means just transforming one schema to another, without the ability to leverage logical operators such as “if,” “then,” “else,” etc. In essence, no logic, just a changing of the data.

• Level 2 Simple transformation and routing, with logical operators (e.g., If this, do this, etc.). You get the idea. We now can peek into messages and make calls based upon content, lookup or even externals such as time and date.

• Level 3 Complex transformation dealing with multiple points and complex schemas and semantic management. Here we are managing one or more schema sources, and one or more schema targets. We also deal with the notion of complex transformation (subjective, I know), but typically that means nested transformations and complex logic, including entire programs that are attached to a transformation.

• Level 4 All of the above, with transformation and flows bound to processes. Here is where we introduce the notion of processes, or the ability to bind information flow, transformation and logic to a process, meaning sequencing, flow control, logical operators, etc.

• Level 5 All of the above, with information bound to services, very simple orchestration (service-oriented process...integration really). Also, no use of process/orchestration standards, such as BPEL or Choreography.

• Level 6 Orchestration or Choreography, or the ability to abstract services for the production of solutions. This is where we deal with service-oriented points such as Web services, providing a binding application of sorts to combine behavior into a solution layer.

• Level 7 Level 6 with data abstraction. This means we are not only dealing with abstract services, but abstract data layers as well. Thus, we’re able to recast many physical databases through single or multiple abstraction levels.

• Level 8 All of the above, with the notion of complex service abstraction and management added in. This means we not only have the ability to create composite processes and orchestrations, but complete composite applications as well that may or may not leverage the abstract data layer.

Consider these as layers, with the higher layers depending upon the lower layers.

What’s important to consider here is not the levels, but what happens as you move up the stack. Information integration is a bit tricky when it comes to SOA, and many don’t consider it, nor create a SOA around an information-only approach. It’s easy to do, considering that ESBs are, in essence queuing systems with service interfaces. Thus, are they transactional/behavior-oriented, or information-oriented? Basically, you’re using services to access information.

As you move forward with building a SOA, you need to keep in mind that you begin with a semantic understanding of your domain, and then move up through the layers to build data abstraction and data services for access to the data/information. Those who approach SOA this way have a tendency to succeed, considering that they are getting a handle on the data, in essence, creating a nice, logical layer before just building services on top of the data.

In contrast, those who don’t consider the information and just layer services on top of dysfunctional and poorly structured data will end up with a very inefficient architecture. It will still be something that has to be addressed at some point in the future. You don’t want to be that guy.

David S. Linthicum is a managing partner at ZapThink. Reach him at david@zapthink.com.

Share this link: http://www.sdtimes.com/link/31466

Integration Watch: The quickly changing market for continuous integration Continuous integration is gaining more acceptance, but as a consequence it means that some platforms will be the focus...

IBM updates InfoSphere for data integration Big Blue has announced two new editions of InfoSphere products that address product information management and data integration. Support for internationalization plays heavily in the data integration release.

Jitterbit 2.0 focuses on reuse in integration The update to the company's open-source integration platform features improvements to its user interface and integration server. Pipeline customization, which allows the creation of reusable scripts in the integration process, is also included.

Add comment

Name*
Email*
Country

Comment
Preview

News on Monday
more>>
SharePoint Tech Report
more>>

Download Current Issue
ISSUE 3/15/2010 PDF

Need Back Issues?
DOWNLOAD HERE

Receive the print Edition?

Google Code turns 5
Google Code Turns 5, and adds a Paxos Algorithm to make the system more stable and reliable.
03/17/2010 11:16 AM EST

Test your Visual Studio 2010 know-how
Microsoft is offering free beta certification exams for Visual Studio 2010.
03/17/2010 11:08 AM EST

Microsoft lifts the hood on IE9
Microsoft is previewing IE9.
03/16/2010 01:10 PM EST