Do-It-Yourself Storage Tools

Masters of high-end storage often devise their own tools and strategies. Here are a few of their secrets.

Interland Inc., a Web hosting firm in Atlanta, decided to build, rather than buy, a tool to track space usage for its more than 78,000 small to medium-size Web sites. Seventy percent of the sites reside on Windows NT servers and the rest are on Linux. A 6 terabyte (TB) XP256 RAID system from Hewlett-Packard Co. provides the main storage.

So, what's going on here? Tools for monitoring and reporting disk space usage for Windows NT servers have existed since the operating system first came on the market.

"We looked at a lot of off-the-shelf tools, but they didn't offer the scalability to support an application service provider environment. So we built," says Robert Malally, Interland's chief technology officer.

Load balancing. Capacity optimization. Storage quota management. Server availability. When it comes to basic storage management applications, many e-storage services have decided to build rather than buy, even from respected storage management vendors.


Inside Your E-Storage Service's Tool Kit

How do you know if your e-storage service offers a bulletproof environment for managing Web storage, as well as for optimizing storage capacity for maximum performance? Make sure it has the tools to do the following:

• Easily locate what storage hardware (such as physical disks) and software (such as remote backup service applications) exist within the organization.

• Know how much capacity is being used at any time and track such usage to forecast needs and plan storage strategies.

• Configure a disk subsystem among multiple servers on different platforms and detect and correct hardware and software problems (both locally and remotely).

• Automate the procedure for physical movement of outboard data applications such as data warehousing or for moving items between storage media.

• Provide central, real-time notification of the exact nature of errors in a network and maintain a record of those problems.

• Monitor on- and off-site tape, optical or other media.

• Provide an ongoing view of the application, server and subsystem performance to spot problems not apparent through other. more granular disciplines.

• Specify rules or policies for managing hardware, files, users, schedules and media. - Elizabeth Ferrarini

The Internet has redefined storage management requirements. Information technology departments now hand off e-commerce storage to outside services, ranging from regional Internet service providers to Fort Knox-like data centers. These e-storage services must provide extreme storage scalability, almost 100% system availability and flexibility to manage growing collections of storage resources.

John Webster, a storage analyst at Illuminata Inc. in Nashua, N.H., says one e-storage service has seen its storage capacity growth rate go from 1TB per month to 1TB per week. "A lot of storage management tools on the market just can't handle this kind of growth. E-storage services can easily justify building management tools for the most mundane tasks," he says.

Homegrown Foundations

Although the company has reached a level of success, as indicated by its $200 million in funding to date, HarvardNet Inc., a Boston-based Digital Subscriber Line (DSL) provider and Web hosting service, has continued to update and use its homegrown tools for managing storage on Windows NT and Sun Solaris servers.

Brian Durkin, director of systems engineering at HarvardNet, says some of those tools date to 1996, when the company's hosting business began. For its shared-server service, HarvardNet uses its disk utilization tools on Windows NT servers to monitor file growth on disk drives, and on Unix systems to monitor the file system for growth, server problems and growth patterns in a customer's space.

HarvardNet's expansion plans include opening one 50,000-square- foot data center in New York, a second in Philadelphia and a third, possibly in Washington. "To support the DSL and the Web hosting businesses, we'll continue to augment our homegrown tools while we evaluate other storage solutions, such as storage-area networks," Durkin says. "We are looking at building advanced monitoring tools to handle our capacity plans."

Some e-storage services have turned themselves into superluxury data hotels for Fortune 1,000 firms that don't want to haul their servers off to a data center and then manage them remotely. Calling themselves "netsourcers," these one-stop-shop online services have built a complete IT infrastructure, including data centers with server and storage hardware (and even spare parts), switching equipment, IT personnel and storage tools to manage everything. Building vs. buying has paved the way for some of these e-storage services to set up shop and maintain business.

Intira Corp., which has three U.S. data centers, has built its business for Fortune 1,000 organizations that want not only to outsource e-commerce but also to provide a good home for other business applications. Founded in 1998, Pleasanton, Calif.-based Intira has stocked the centers with storage products from vendors such as EMC Corp., HP, IBM and Sun Microsystems Inc. but manages them largely with an array of homegrown tools.

For servers and storage, John Steensen, chief technology officer at Intira, says his company has built an extensive, real-time system (layered over HP's OpenView) that monitors 400 data points so customers can be assured of server availability and adequate storage capacity. If a customer's storage exceeds a certain threshold, then Intira carries out the customer's escalation procedures for allocating more space through load balancing.

"We built because there weren't any management suites that went from the bottom of the network to the top. We didn't want to buy tools and glue them together into a coherent picture for our customers," Steensen says.

Doubts About Do-It-Yourself

Another netsourcer, GlobalCenter Inc. - Global Crossing Ltd.'s Web hosting division, which came with the acquisition of Frontier Corp. - has built up its IT infrastructure (which includes eight data centers) by riding on GlobalCenter's network and through a combination of buying, building and forming partnerships.

Jason Schaeffer, director of systems and infrastructure at the Sunnyvale, Calif.-based company, acknowledges that the firm built a lot of storage management tools during its early years in 1996 and 1997, but he says GlobalCenter is getting out of do-it-yourself storage tools. "We're not looking to do that anymore. We're getting the key storage management tools we need either through partnerships or acquisitions," he says.

Schaeffer says a lot of his competitors have built their own storage management tools, but he no longer thinks it makes sense for his company.

"Developing storage management suites doesn't make sense, especially if it's not your core competency," he says.

Managing storage capacity on servers hasn't turned into a juggling act for GlobalCenter. "We don't allow our capacity to exceed a 40% usage rate. If it does, then we go to fail-over," Schaeffer says.

Carl Howe, a research director at Forrester Research Inc. in Cambridge, Mass., says some of the management infrastructures that e-storage services have built "may be prone to sporadic crashes for no apparent reason or staffed with individuals who don't understand how to resolve problems."

Another analyst points to a widely reported 30-minute outage at Charles Schwab & Co.'s Web trading site last year that left thousands of customers fuming. Schwab was trying out a new storage management product it had developed for setting up a dedicated space for DB2 transaction logs. The idea was to keep the logs from filling up other storage areas. Ironically, the new product generated a large log file that filled up a partition. When the storage management application tried to access data on the disk, the disk locked, bringing down the entire Web site.

The outage could have been avoided if Schwab had built a tool for detecting how quickly partitions filled up with log files, but that development was overlooked as the tools were developed in-house, says Webster.

Perhaps inevitably, building storage tools in-house has given some developers the notion to take their ideas to market.

Chris Hickson and Steve Anderson - both veterans of Microsoft Corp.'s Microsoft Network (MSN) online service - parlayed their knowledge of load balancing and experience in building load-balancing tools to raise $7 million in venture capital to set up Viathan Corp. in Seattle. This start-up solves one key problem faced by many online services: how to allocate Web database storage dynamically.

The duo joined MSN when it was a proprietary service with about 1 million users. Anderson, Viathan's CEO, says, "We were working on scaling initiatives for what we thought were a lot of users. Microsoft then opened the floodgates, and we had our work cut out for us." Once it was opened to the Web, MSN jumped to 4 million users within a few months. Microsoft's Hotmail acquisition took MSN to 20 million users. "Very few companies had seen this type of scaling," Anderson says.

He says scaling at the application server layer was easy to do. But scaling broke at the back-end SQL Server database layer. Hickson and Anderson wore their fingers out writing code to get back-end databases to accommodate armies of new users and to keep track of new information about each user. They also got tired of being available around-the-clock to wake up servers that took a siesta.

They found that e-commerce relational databases yield two types of information: catalog information, which grows statically, and user information, which grows exponentially.

" may add 200 new titles to its book catalog daily but add 5,000 new users each day, along with information about current users. The database has to keep track of each user's purchases, shopping-cart history and pages clicked on," says Anderson. Storage management tools can't handle the massive scaling required by this type of Web data, he says.

The duo built the Viathan load-balancing software for Web sites such as Hotmail and established e-storage services that maintain user information in back-end SQL Server databases. The software virtualizes back-end databases across a cluster of thin Windows NT servers. The software handles the partitioning, clustering, replication and fault tolerance for the entire system.

Anderson says that many storage management issues for the Web have yet to be solved and that some of the answers are being developed in IT shops right now to address specific problems. "IT professionals shouldn't hold back their entrepreneurial ambitions, especially if their storage management tools can satisfy a universal customer need," he says.

Ferrarini is a freelance writer in Arlington, Mass.


Copyright © 2000 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon