LOGIN | REGISTER NOW | SUBSCRIBE
 
 
News on Monday
more>>
SharePoint Tech Report
more>>


   

 
 
Download Current Issue
ISSUE 2/1/2010 PDF

Need Back Issues?
DOWNLOAD HERE

Receive the print Edition?


 
blogs tab
Visual Studio 2010 Release Candidate Available Today
A Visual Studio 2010 release candidate is available on MSDN.
02/09/2010 09:45 AM EST

Is Microsoft eyeing Office subscription pricing?
Microsoft may be preparing to offer a new Office pricing option called "union," which charges the same for cloud as on-premises.
02/01/2010 09:38 AM EST

Facebook rewrites PHP runtime
Facebook is about to open source its own PHP runtime, written from scratch for speed.
01/30/2010 08:53 PM EST

 

Events calendar tab
Macworld 2010
2/9/2010 to 2/13/2010
San Francisco
IDG World Expo

SPTechCon 2010
2/10/2010 to 2/12/2010
San Francisco
BZ Media

PyCon 2010 (Python Conf.)
2/17/2010 to 2/25/2010
Atlanta
Python Software Foundation

Southern Calif. Linux Expo
2/19/2010 to 2/20/2010
Los Angeles
SCALE

IBM Pulse
2/21/2010 to 2/24/2010
Las Vegas
IBM


 
print Printable version 
Most Read Latest News Blog Resources
Facebook brings HipHop to PHP
HipHop compiles PHP to speed up the Web layer at world's most popular social networking si...

Adobe moves LiveCycle application platform to the cloud
Release gives developers just what they need to create targeted business process-oriented ...

Oracle acquires AmberPoint
Today's merger will probably affect most of AmberPoint's existing partnerships with other ...

Digg!  Digg
Reddit  Reddit
Slashdot  Slashdot
Share on facebook  Facebook
Share on Twitter  Twitter

David Rubinstein: Data Streaming Crosses the Chasm


By David Rubinstein
December 15, 2007 — 
This is the time most publications wax nostalgic over the year that was, recounting the highlights and lowlights, and trying to put it all in perspective. Count us among them; SD Times will publish its traditional "Year in Review" issue Jan. 1.

So, what to write about in this, the final issue of 2007? Why, what we can expect in 2008, of course. You've got to see where you're going to know where you've been…or something like that!

Anyway, with the help of Daniel Chait, founder of software consulting company Lab49, here are some recent trends that are poised to cross the chasm, pass the tipping point and take off into the everyday world of software development.

The first area that will become much more commonplace in 2008 is data streaming, otherwise known as complex event processing. CEP, which had been relegated to highly specialized applications such as financial trading, will make its way into retail and other markets that have a need to perform calculations on data and see patterns in near real time.

The financial services industry has driven CEP, because institutional, algorithm-driven trading has led to massive increases in trading volume. "The old method of store and process for data is falling by the wayside," Chait said. "Writing processes in batch jobs at the end of the day is increasingly inadequate." Financial services companies need information on activity, trading and market information throughout the day to remain competitive.

Complex event processing came out of the world of academia, Chait noted, where the work was focused on processing data from sensors. In the area of weather prediction, sensors that can collect data on temperature, humidity and wind speed need to be read in real time to be useful.

For retail, Chait said companies can analyze traffic and consumer patterns. "Using CEP engines, they can do clicksteam analysis, to see what's going into a shopping cart, what add-ons are being chosen, when dropouts are occurring," he said. A company might use that data analysis to decide to mark down a certain item on its Web site for a certain time period of the day to drive sluggish sales, for example.

This need to handle massive amounts of data and transactional information will give rise to wider implementation of parallel and distributed computing, Chait believes—the second trend that he sees coming into its own in 2008. "This type of massive data streaming is beyond the capabilities of the fastest computers out there," he said. "We need distributed data caching and grid computing to handle it."

The use of multicore processors is creating challenges for developers looking to take advantage of the increased computing power. "It's not simply a given that an application will run twice as fast on a dual-core machine," Chait noted. "The application has to be architected" in a way to best utilize such features as failover and provisioning. The success of departmental grid projects is forcing enterprises to look at grid computing and management, such as when to bring in additional hardware or when to take a server out.

Microsoft has come out with utilities around parallel computing, Chait pointed out, to enable developers to write applications in the way they're most comfortable, and have the machines do the parallelization of the applications.

So, with increasingly large numbers of events occurring in these systems, and multicore processors being created to handle the burgeoning load, what must follow is a way to make sense of all the data being pumped out of these systems. That, according to Chait, will lead to a growth in the data visualization market in 2008.

"Developers need to think about new ways of conceptualizing and presenting these massive amounts of real-time data in a way people can grasp and make sense out of, and have it delivered over the Web on a browser," Chait said.

Web applications will continue to look and act more and more like client/server applications, with all the interactivity, charts, animations and multiple windows.

He believes Microsoft, with its Windows Presentation Foundation technology for next-generation graphics display, will have the greatest impact because of the company's reach, and added that Adobe, with its Flex technology, will continue to make inroads and gain market share.

Look for such things as 3D graphics and transparency to have greater uptake in the year ahead, Chait added.

But before then, we'll take a look at how we got here with a review of 2007. Happy holidays, everyone, and we'll see you back here next issue—and next year.

David Rubinstein is editor-in-chief of SD Times.


Share this link: http://www.sdtimes.com/link/31419
 

Add comment


Name*
Email*  
Country     


  • Comment
  • Preview
Loading