|
|
AS OF 5/17/2008 4:10AM EST
|
Adjusting Testing for SOA
Planning, tools, funding key to solving complex problem
By David S. Linthicum
August 15, 2007 —
The need to figure out new ways to approach testing in the world of service-oriented architecture takes center stage as SOA deployments increase exponentially. To test SOA in the real world, not much changes, and a lot changes at the same time. We don’t toss out traditional testing approaches, techniques and tools, but we need to rethink the concepts and technology behind SOA and adjust accordingly.
Testing SOAs is a complex, distributed computing problem. You need to learn how to isolate, check and integrate, assuring that things work at the service, persistence and process layers. The foundation to SOA testing is to select the right tool for the job, have a well-thought-out plan, and spare no expense in testing cycles or else risk that your SOA will fail.
Organizations are beginning to roll out their first instances of SOA, typically as smaller projects. While many work fine, some are not living up to expectations due to quality issues that could have been prevented with adequate testing. You need to take these lessons, hard-learned by others, and make sure that testing is high on your priority list when you dive into SOA.
Defining SOA
First, let me put forth my definition of SOA, so we have a foundation.
In short, a SOA is a strategic framework of technology that allows all interested systems, inside and outside of an organization, to expose and access well-defined services that may be furthermore abstracted to orchestration layers and composite applications. The primary benefits of a SOA are an agile architecture that’s changeable around changes to the core business processes, and reuse of services from application to application, domain to domain.
Certainly, this is not the only definition; however, it’s good enough for our discussion of SOA testing.
SOA allows enterprises to share common application services as well as information. This sharing is accomplished either by defining application services that can be shared and therefore integrated, or by providing the infrastructure for such application-service sharing. Application services can be shared either by hosting them on a central server or by accessing them interapplication (for example, through Web services).
Using the tools and techniques of application integration gives us the opportunity to learn how to share common application services. More than that, these tools and techniques create the infrastructure that can make such sharing a reality. By taking advantage of this opportunity, we provide the infrastructure for the reuse of business logic and application behavior.
The uses for SOA are endless, including creation of composite applications, or applications that aggregate the processes/services and information of many applications. For example, using this paradigm, application developers simply need to create the interface, and then add the application services by binding the interface to as many application services as required.
However, SOA is not all about service sharing; it’s also about agility. If you do SOA right, you’ll have an architecture that can change as your business needs change. Typically, this is accomplished by placing volatile things in their own “configurable” domain, such as an orchestration layer that allows you to alter processes to adjust to a changing business, or the ability to mix and match services to form a solution. Thus, when testing a SOA, you must consider the use cases around agility, as well as the stability and validity of the underlying services and data.
Testing a SOA
So, what’s new in the world of SOA as it relates to testing? In a word: services.
At the service deployment stage, we focus on testing and early management. Do the services work properly? Which tools should you use to diagnose problems? How should we approach testing procedurally?
Also, what about governance and security? Both concepts use new approaches with SOA, and have to be considered as well. For now, however, let’s focus on services. Or, what many SOA testers call “testing the core.”
While there are no hard and fast guidelines on what makes up a well-defined and developed service, we do know a few things.
Services are not complete applications or systems. They are small parts of an application. Nor are they subsystems; they are small parts of subsystems as well. Indeed, services are more analogous to traditional application functions in terms of design, and how they are leveraged to form solutions, fine or course. Knowing that, we have a better basis of understanding when approaching the service-testing problem.
Each service has a specific purpose, and they are not complex or naturally dependent upon other services. Thus, they are easily abstracted into composite applications, in essence, leveraging these services as if they are functions local to the composite.
Services should exist with a high degree of autonomy. They should execute without dependencies, if at all possible. This allows you to leverage the service by itself, and design the service with this in mind, no matter how coarse- or fine-grained the service is.
Considering that, when testing services (for example, Web services, Java EE, etc.) you have to think about a few things, including autonomy, integration, granularity, stability and performance, in the particular order of your requirements.
You need to check for autonomy to determine how the service functions on its own, including any dependencies that may be present. This is especially important when the service will typically be leveraged on its own, which most are. Most services do something right once, and that’s their contribution to the SOA solution.
Integration testing is required to see how the service works when leveraged by other systems that perhaps are known or unknown at the time of development. While you may think using Web services standards determines compatibility, those who deploy services already know that nothing is certain unless you test it. Different interpretations of standards among different vendors can cause integration issues.
Granularity testing determines if the service was created with too much of a coarse- or fine-grained leaning, which has an effect on performance and the value of a service. For instance, if a service is too fine-grained, it could cause performance problems as too many fine-grained services are deployed and yakking across the network, or prove not as useful if the services are coarse-grained. Poorly designed services are the most common issues when considering SOA performance and scalability.
Stability testing ensures that the services built won’t fall down at the worst of times. This is usually simple regression testing, with some integration testing here as well. And performance testing is just what you would expect: the ability to determine if the services can handle many simultaneous requests, and any special architecture that may be required to ensure good performance, such as load balancing with transactions.
Performance testing makes sure that you test the services under increasing loads, ensuring that the performance expectations of the service, and the service-level agreement, are met. This means the services are placed inside of a testing consumer where they are invoked, measured, logged and controlled during the testing process.
All About Reuse
So, now that we know the basics, what do you test for within services beyond that of traditional software testing? First, it’s important to follow a few basic principles.
First and foremost, services should be tested for reuse (reusability). Services become a part of any number of other applications, and thus must be tested so they properly provide behavior and information, but not be application- or technology-specific. This is a difficult paradigm for many developers since custom one-off software that digs deeply into native features is what they’ve been doing for most of their careers. Thus, the patterns must be applicable to more than a single problem domain, or application, or standard, meaning you must have use for your reusable service and it must be in good working order.
To test for reusability you must create a list of candidate uses for the service. For instance, you might have a shipping service that plugs into accounting, inventory and sales systems. Then, the service should be consumed by the client, either through a real application (in a testing domain) or a simulator, and the results noted.
In addition, the service should be tested for heterogeneity. Web services should be built and tested so that there are no calls to native interfaces or platforms. This is due to the fact that a service, say, one built on Linux, may be leveraged by applications running on Windows, Mac OS and even mainframes. Those that leverage your service should do so without regard for how it was
created, and should be completely platform-independent. The approach to testing this is rather obvious; simply consume the service on several different platforms, and note any calls to the native subsystems.
You should also test for abstraction. Abstraction allows access to services from multiple, simultaneous consumers; hiding technology details from the service developer. The use of abstraction is required to get around the many protocols, data access layers, and even security mechanisms that may be in place, thus hiding these very different technologies behind a layer that can emulate a single layer of abstraction.
Abstraction is tested effectively by doing, which means implementing instances and then testing the results. Regression and integration testing is the best approach, from the highest to the lowest layers of abstraction.
When we build or design services, we need to test for aggregation. Many services will become parts of other services, and thus composite services leveraged by an application. You must consider this in their design. For instance, a customer validation service may be part of a customer processing service, which is part of the inventory control systems. Aggregations are clusters of services bound together to create a solution, and should be tested holistically through integration testing procedures.
Breaking Services Apart
Services should be white-boxed like an application, but they should deal with three major subsystems: the interface, the data and the behavior. This is the core of your SOA—the resources.
In the world of SOA, there are two types of services: data services and transactional services, and each requires a different approach to testing.
Data services, as the name implies, means that the service typically provides data abstraction representing a physical database or databases, and thus is more data-oriented and should be tested as such. You need to monitor the consumption of information from the physical data stores, the remapping of the data into the new schema, and the new structure provided to the other services or orchestration layer that is interacting with the data service.
Transactional services deal with data and are really the core services providing application behavior and functionality. Thus, while you do need to monitor information flowing through these services, and the changes made to that information, you also need to pay attention to the processing, and test those features as well.
Dealing with the interface is very similar to dealing with an API. While it’s typically SOAP interaction as defined by WSDL, in some instances you’re using REST (Representational State Transfer) or proprietary enabling technology. The approach is the same. It’s really an input/output interaction, thus the programmer needs to build in breaks or output calls that display the information and parameters as consumed by the service during execution.
Testing of the data as flowing through a service is almost identical to testing a traditional application. Once again, monitoring points are leveraged to watch the data as it flows through the service. Also, consider the information the service consumes, with the information the service produces.
Finally, we need to focus on the behavior, or the functionality of the service. This means that the logic of the service is monitored and determined to be sound, and that the information flowing into the service is processed correctly as to the design of the service. Moreover, if the service is designed to provide different behavior via context, that needs to be monitored throughout the processing of the service as well. Same approach: Set up monitoring points, and watch the behavior through execution.
Testing SOA in the real world means understanding and integrating new testing methodologies to account for SOA services, governance and security.
Traditional testing methodologies must still be a part of SOA testing, but we cannot limit our SOA test cycles to these traditions. Moreover, we cannot allow our thought processes and experiences from previous test environments to rule our actions when we move into SOA. The success of your SOA project can and likely will depend upon how effectively you integrate and adjust your test cycle for the concepts and technology behind SOA.
David S. Linthicum, who writes the SOA Watch column for SD Times, is the CEO of the Linthicum Group. Reach him at david@linthicumgroup.com.
|