Opinion: Grid an option for data management challenges

Steve Tuecke, Univa   Today’s Top Stories   or  Other Hardware Stories  

A Roundtable Discussion on File Management Getting the Most from DFS Bringing Order and Security to your Mobile Workforce: Corporate Mobility Policy and Device Management Symantec Backup Exec" System Recovery 8.5: The Gold Standard in Complete Windows System Recovery Fast and Simple Recovery of Your Critical Microsoft® Applications with Symantec Backup Exec" Breaking through the Dissimilar Hardware Restore Challenge Virtual Reality SCSI Inflection Point: The New Era of Serial Attached SCSI SaaS Solutions for Remote Systems Management Sign up to receive Storage Resource Alerts

February 23, 2006 (Computerworld) -- With EMC Corp.'s acquisition of Acxiom Corp.'s grid computing software for $30 million last month (see "EMC Partners With Acxiom to Build Grid-based BI Systems"), enterprise customers started opening their eyes to the fact that grid is not just about raw horsepower and CPU utilization for high-performance computing environments.

So what was it that Acxiom did so well with its grid environment that caught EMC's attention? To put it simply: data management.

Acxiom has a very popular data-integration application called AbiliTec. It took the "scale out" commodity hardware route to scale and support a growing number of transactions (as Google Inc. and Amazon.com Inc. have done) and then built its own grid software to manage this new environment. In an article on Acxiom's environment last year, Computerworld reported that its grid had grown to 6,000 Linux nodes, processing more than 50 billion AbiliTec transactions per month (see "Case Study: Acxiom Corp.'s Homegrown Grid").

Performance and reliability have been at the heart of Acxiom's data management grid story, but there are some other very specific enterprise data challenges where grid has already been used in research and science. Today, enterprises are increasingly evaluating the capabilities of grid infrastructure to resolve data management issues ... above and beyond data processing horsepower.

Transporting Massive Amounts of Data

Your typical enterprise is probably not going to be dealing with data on the petabyte (1 quadrillion-byte) level any time soon, like particle physicists in the online science realm do today.

However, many commercial entities do transport enormous files on a daily basis. Consider cases like the British Broadcasting Corp., where one hour of preprocessed high-definition broadcast averages about 280 gigabits. These organizations are working with grid technologies today to make their data assets accessible to field reporters and users across a distributed network.

Moving large data sets at high speeds between distributed sites is a common challenge in many industries. Oil and gas companies are perhaps the poster children for moving large data sets, which they accumulate through seismic analysis and reservoir analysis. Getting the "whole picture" to make sound business decisions requires pulling large quanta of data from many different locations.

Other markets with massive data-transport requirements include the automotive industry (for computer-aided analysis and simulations), semiconductor companies (for mask layout based on instruction sets) and pharmaceutical firms (for molecular matching and chiral synthesis), to name just a few.

Getting Data Out of Complex Storage Systems

Grid pros have popularized the expression that "access to the data is as important as access to compute resources." Sometimes in enterprises, the challenge with data access -- beyond the size of data sets -- is the complexity of the protocols associated with storage systems.

Continued...
1 | 2 | NEXT  

Print this Story

Send Us Feedback

E-mail this Story

Digg this Story

Slashdot this Story

"The much-maligned Microsoft Zune apparently has the world's most powerful man as its fan. President-elect Barack Obama was seen using..." Read more... "In Thursday's..." Read more... Read more Hardware posts or See all Blogs

Windows users indifferent to Microsoft patch alarm, says researcher Tech jobs down sharply but not out Apple yanks antivirus advice from its Web site More top stories...

Microsoft slates 8 bug updates for year's final Patch Tuesday De Beers tries to force spoof news Web site offline over fake ad Microsoft confirms Yahoo's Lu to run online services

Review: The new MacBook Air, now with extra SSD goodness Thin as ever, the latest Air offers up to twice the storage and snappy performance. Cool Stuff: Your 2008 Holiday Gift Guide We've got an array of economical, expensive, and just plain weird tech gifts for your friends and family. Massive botnet returns from the dead, starts spamming The spam-spewing 'Srizbi' botnet that was shut down two weeks ago has been resurrected and is again under criminal control, say security researchers. Elgan: Why you can't trust 'friends' on Facebook Facebook is popular and growing -- especially with criminals. Here's why they love it. Windows 7 Get the latest news, reviews and more about Microsoft's newest desktop operating system More Continuing Coverage Windows 7 Voting Technology Google's Chrome browser Economy in turmoil: Latest news and tips iPhone 3G Bill Gates moves on Windows Vista A to Z Mac A to Z 2008 Salary Survey Find wage data for 50 IT job titles. More Special Reports 2008 Premier 100 IT Leaders 2007 Premier 100 IT Leaders 2008 Best Places to Work in IT 2007 Best Places to Work in IT 2008 Salary Survey 2007 Salary Survey 2006 Salary Survey 2008 IT Forecast 2007 IT Forecast 40 Years of Computerworld Top 12 Green-IT Users The FAQs About Going Green IT Schools to Watch iPhone: One Year Later Global Mobile Your Next Data Center Management: Reinventing IT Stop Data Leaks Web 2.0 Security Surviving Disasters Managing Storage Virtualization The Lean Storage Machine Storage on Trial Can Web 2.0 Save BI? Bypass These BI Potholes All Zones Business Continuity Zone The File Data Management Zone Security Management Zone The SAS Zone Business Intelligence and Analytics Zone The Enterprise Search Zone Software as a Service Zone The Security Zone See your link here

Enabling Data Centers that Are Both Automated and Dynamic Enabling Data Centers that Are Both Automated and Dynamic View this webcast now! Go to the webcast  The Business Value of Tape Storage Download this complimentary Technology Briefing today! (Source: Dell) Reliable. Long-lived. Portable. Affordable. Energy-stingy. These and many other attributes of tape storage are virtually assuring a continued market for tape storage, a conclusion reached by IDC and many others. Combined with other storage solutions, tape is an integral part of any data archiving strategy. Download this executive briefing  Brocade and the File Area Network - A Taneja Group Solution Profile Get this white paper now! (Source: Brocade) This Taneja Group report examines how Brocade FAN solutions are creating a stateless end-to-end file and block data infrastructure. Download this white paper  White Papers Read up on the latest ideas and technologies from companies that sell hardware, software and services. 8 Things You Need to Handle Today's Network Traffic Next-Gen Load Balancing: 3 Keys to Successful Delivery of Advanced Web Apps Building a Reliable and Dynamic Data Center with PAN Manager by Egenera View more whitepapers

• Windows users indifferent to Microsoft patch alarm, says researcher
• Microsoft slates 8 bug updates for year's final Patch Tuesday
• Tech jobs down sharply but not out
• More top stories 

Business Continuity Zone An organization's business continuity plan helps keep critical functions running during an emergency–the power fails, a virus is unleashed on your network, a natural disaster has occurred. Even the slightest downtime or loss of data can cripple your operation. CDW can help you prevent disaster by implementing a well-planned recovery strategy. Click here to visit the Zone See All Zones

Intercept Spam & Viruses With MessageLabs MessageLabs is offering a complimentary 30 day trial of its managed Anti-virus and Anti-spam security solutions. MessageLabs guarantees complete protection against all know and unknown email threats. By providing 24 hour support, your business can increase productivity and decrease risk. Register for a complimentary trial and receive a free datasheet. Download this white paper now!

Jack Demo Have WAN acceleration solutions got your head spinning? Our Technology Demo sorts it out. See how our technology offers a comprehensive solution to the entire WAN performance issue including mobile workers' needs. Download this demo and discover how everyone, from branch office workers to executives and sales people on the go, get the same LAN-like access to their enterprise applications and data they enjoy when they're at headquarters. Go now!

• Too good to ignore: 6 alternative browsers

• Apple yanks antivirus advice from its Web site

• Microsoft spells out Vista SP2 contents

Read more news 

• Myth: Every system has Internet access

• Christmas Lottery

Read more Shark Bait