Acxiom Corp. processes billions of records every month, culling from sources like customer-prospect lists, phone records and retail store sales to generate usable consumer data for its business clients.
The high volume was impressive, but company officials wanted to process even more -- quicker and cheaper, too.
"We decided there had to be a better way," says Charles C. Howland, grid infrastructure group leader.
So tech workers developed the Customer Information Infrastructure (CII), winner of a 2005 Computerworld Honors award. This grid environment allows Acxiom to handle a higher data volume faster and with less-costly equipment.
Consider, for example, that it often took more than three months to update Acxiom's InfoBase database; on the grid, it takes three days. "We would not be able to run our business the way we do today without this capability," says Alex Dietz, leader of the Acxiom solutions infrastructure organization.
Acxiom had managed most data using IBM mainframes running MVS until 1995, when it moved its internal processes and clients' applications to symmetrical multiprocessing platforms. Although SMP technology was more powerful and cost-effective, Acxiom still spent more than $150 million annually for capital equipment to maintain its capability.
But Acxiom staffers were already at work developing a high-performance application called AbiliTec to link and clean information on individual consumers gleaned from multiple data sources. Acxiom matches every name and address it receives from clients against its in-house AbiliTec reference base of 20 billion records. More than 40 billion records are linked each month.
 |  |
Acxiom's Charles D. Morgan, Alex Dietz and Terry Talley Image Credit: Kelly Quinn | |
The application worked well, but Acxiom needed 20 environments with Unix SMP supporting AbiliTec to handle the processing load. It was expensive and still not fast enough, says Terry Talley, a senior technical adviser based in Conway, Ark.
Payback Potential
So in 2000, a research team set out to find a better way, pinning its hopes on grid technology. Dietz credits Talley with the plan: "He came to us with the idea of wiring together a bunch of PCs, and he proved it would work."
Talley says he spread applications over multiple machines "instead of using one big machine. We were much faster, and the incremental cost to do one record was significantly lower than our previous implementation."
The team of eight to 10 tech workers worked on and demonstrated the grid computer project to CEO Charles D. Morgan in the summer of 2001.
"He said, 'This is great. Go do it to all of the Acxiom products,' " Talley recalls.
Acxiom's use of grid technology makes the company a leader in this area, says Ahmar Abbas, an analyst at Grid Technology Partners in South Hadley, Mass., and author of Grid Computing: A Practical Guide to Technology and Applications (Delmar Thomson Learning, 2003). As for the technology's impact on Acxiom's performance, he says, anything workers can do to make processes run better, cheaper and faster "is going to have a direct impact on the services they offer and the money they can generate."
John Ripa, group leader for Acxiom data products, says the impact of the new technology is significant.
He points to one of the company's products, InfoBase Enhancement, as a prime example. A client -- a cell phone company, for example -- might ask for consumer information to target new customers or to cross-sell to existing ones. The client sends Acxiom millions of its own records, which Acxiom then processes against its database of consumer information to produce the detailed consumer files the cell phone company wants.
Working with the CII grid computing technology, Acxiom improved the speed of its build process by 83%, Ripa says. It increased the speed at which it delivers these files to clients by 77%. "And the reliability improved dramatically. We're as close as we can get to zero downtime," he adds. Equally impressive is an 86% reduction in hardware costs, Ripa says, comparing costs prior to and after implementation.
"This gives our clients the ability to do things rapidly that could never be considered before," Ripa says, adding that companies are willing to pay a premium for that speed.
Talley says the biggest challenge for Acxiom was "dealing with the psychological impact. People are comfortable with paradigms that are old and familiar." The changes required workers "to rethink existing processes and software." Acxiom also had to manage a large number of computers over the long term.
"We have built a lot of software to address this challenge," Talley says. "It's relatively easy to get a bunch of machines up and running for the first time. It's much more difficult to add to, replace and update those machines over time, and the problem is magnified if you have thousands of nodes."
Lessons Learned
Without a road map to guide them, Acxiom's IT workers had to rely on their own internal resources to compensate for a lack of commercial products. As a result, they built their own resource scheduler, grid control, maintenance interfaces, software distribution functions and grid-enabled data management functions.
Open-source software was used when available; when it wasn't, the IT staff wrote components. Acxiom officials also tapped experts who were developing general-purpose grid products at other companies to confirm that they were on the right path.
The rewards overshadow many of the challenges. Talley points to a demographic enhancement product that took nearly 30 days to run on a large Unix computer; it takes less than one day on the grid version.
"Our grid is all about performance. It's about being able to do things you couldn't do before," Talley says.
In 2003, Acxiom announced that it would host client data and run client processes in the grid environment, too -- a strategy that evolved into the architecture known as CII.
CII product leader Ken Archer says the speed, flexibility and scalability of the grid is key to meeting clients' needs.
"A large part of our customer base is financial services, specifically around customer marketing and customer acquisition. And if they can get the data quicker, they can get offers out faster to make those decisions," he says.
Acxiom now has more than 4,000 rack-mounted, two-processor nodes in its data centers that are dedicated to the grid. Each node is a PC-based server running Linux.
Officials won't disclose how much the company has invested in its grid computing project, although they indicate its value is well worth the cost. They cite the case of one large credit card issuer, which had a file of 250 million customer records processed and scored in parallel using both the CII environment and mainframe; the time to completion with CII was 15 hours versus more than 150 hours on the mainframe.
Dietz says Acxiom is still migrating to grid computing, so about half of its work still flows through legacy environments. Says Talley, "We'll have a constant evolution in both size and function over the next few years."
Pratt is a Computerworld contributing writer in Waltham, Mass. Contact her at marykpratt@verizon.net.