In the Trenches at an ASP

Here's an inside look at how an application service provider works in a market where flawless performance and reliability are a must.

Paying a monthly fee to have someone else buy, configure and manage all the equipment your business uses, maintain your company's network connection to the Internet and troubleshoot the inevitable glitches in both hardware and software sounds like a way to eradicate many information technology headaches. At least that's what application service providers (ASP) are banking on.

During the past year, the nascent ASP market has attempted to take on the hosting and management of many complex applications, for both small and large enterprises. The number of headaches they've alleviated, however, has yet to be determined.

Renting can have its downside, too: Customers don't define how an ASP avoids trouble, and they don't control its responses to problems; that happens in the data center. So if you're going to use a service provider, you need to carefully check out its operations. With this in mind, Computerworld visited NaviSite Inc.'s data center in Andover, Mass., to see what we could learn about the ASP business.

Row upon row of black racks and boxes are arrayed inside NaviSite's 20,000-square-foot data center. The center, which opened in January, will eventually grow to 52,000 square feet, according to company officials. NaviSite, just three years old, outgrew its first center last year.

NaviSite started out as the internal IT department of CMGI Inc., a venture capital group also based in Andover. Back then, the group hosted and managed many of CMGI's Web businesses and gained expertise in finding and solving problems in Web applications that were becoming increasingly complex.

CEO Joel Rosen says the company is capable of performance management because "NaviSite cut its teeth working with sophisticated businesses."

NaviSite specializes in e-commerce applications. It provides a fixed-network architecture that's built using routers and switches from Cisco Systems Inc.; FireWall-1 from Check Point Software Technologies Ltd.; storage equipment from EMC Corp., Compaq Computer Corp. and Dell Computer Corp.; and load-balancing equipment such as a switch from ArrowPoint Communications Inc. or Cisco's LocalDirector product.

Customers can rent whatever equipment is appropriate to their needs, leaving management and maintenance to NaviSite. A few customers rent the equipment from NaviSite but choose to operate and maintain it themselves.

NaviSite also provides software such as Sun Microsystems Inc. or Windows NT servers, Oracle Corp. or SQL Server databases, Allaire Corp.'s ColdFusion Web application server and SilverStream Software Inc.'s Application Server. If customers choose one of the primary software offerings, NaviSite can provide a range of performance management services, from preventive maintenance to on-the-spot repairs. If NaviSite doesn't know a specific piece of software well enough to perform all maintenance itself, it won't guarantee that level of service but will still locate any problems that arise and help coordinate solutions. NaviSite knows that its success depends on keeping the power on, the applications running and all connections open to its customers.

Powerful Preparation

Sidney Kuo, NaviSite's product line manager and a mechanical engineer by training, points out details in NaviSite's data center with an engineer's pride. Beneath the raised floor of the data center, a 24-inch crawl space (double the requirement) makes it easy to run and fix the wiring that connects applications to the Internet.

In the rear of the data center stands a row of black cabinets that funnel electrical power to the systems. "Electricity is key," Kuo says simply. "Without it, nothing runs, and that would be a problem."

A few weeks before Computerworld's visit, a motorist hit a utility pole on a nearby road and knocked out power to the building for the first time since the new data center went into operation. Kuo watched vapor from the backup generators begin to appear just 10 seconds after the cafeteria lights went out. Electricity was restored later that day.

In the event of an electrical outage, battery power takes over immediately, giving the four diesel generators a chance to warm up. Combined, the generators can generate 2,500 kilowatts of power - enough to keep the center going indefinitely, as long as there's a steady supply of diesel fuel. In the event of a more devastating power outage, NaviSite has a second data center on the West Coast that operates on a separate power grid. The company refers to this as "N+1" redundancy: batteries, electrical generators, backup generators and separate power-grid coverage.

Next to the electrical panels stand locked cabinets containing each customer's equipment. Each cabinet is marked with a small white label in the upper left corner that indicates its owner or the company using it.

Although Kuo is NaviSite's product line manager, he doesn't have a key to the data center. A security guard has to open the door for him. Only employees who need daily access to the data center have entry privileges, and even they are carefully monitored.

To verify identity, employees run their badges through a card reader and their hands across a palm reader. Even then, they can't get access to customer equipment. Only the guard and the customers have keys to customers' cabinets. And when customers come to the data center, they must tell the guard their passwords to gain access.

In the data center, Kuo pulls out what looks like a football-shaped key ring with a digital face. It's actually a random-key generator that changes and coordinates another set of passwords for the network architecture. Without the updated number, no one has access. The point is to ensure that only a limited number of people can touch or alter anything, especially the core routers that connect NaviSite to the Internet.

The routers, Cisco 6509s, sit in a locked chamber called the main equipment room. The room, like everything else in the data center, exists in duplicate. On the other side of the building, there's another room that contains the exact same setup. The main equipment room is where NaviSite routes Internet traffic.

The fiber-optic cables that connect the data center to the Internet enter at four different locations. It's a hedge against the risk of "backhoe failure" - the possibility that someone might accidentally cut through the fibers while digging up a sewage pipe.

The on-ramp to the Internet is the most compelling reason most customers use an ASP's data center. NaviSite buys backbone Internet access from the major providers: AT&T Corp., Sprint Corp., Cable and Wireless PLC and GTE Internetworking. It's called a private-transit strategy: By paying for backbone access, NaviSite avoids the free but crowded public-access ramps to the Internet.

Between the two equipment rooms are still more black boxes, which back up the information going through the center. Each box contains 500 tapes, and each tape has 70GB of storage space.

Backup may seem mundane, but a storage problem recently brought down ASP for two days. The San Francisco-based firm had to shut down service while it isolated a problem with its backup devices, and it didn't have a second set of backup equipment that could take over while it fixed the first.

Kuo explains that he could remove any one piece of equipment from the NaviSite equation and the system would continue to function. "If I pulled out a switch, nothing would happen to the operations. The system would reroute traffic," Kuo says. "Each piece of equipment in our system is backed up more than once."

Redundancy - even backup plans for backup failures - defines NaviSite's data center, according to Rosen and Kuo.

Reliability may be essential, but The Dress Barn Inc. in Suffern, N.Y., uses NaviSite because the clothing retailer can rent both hardware and software. Dress Barn has chosen to avoid a heavy investment in equipment as it gears up for Internet retailing. The clothing retailer rents four server licenses from NaviSite.

"It was the service which kept us coming back as we evaluated ASPs," says Chris Correia, Dress Barn's director of IT. NaviSite has kept Correia aware of new technologies and has helped define the software and hardware configurations it will need.

New Technologies, New Expectations

NaviSite has a track record of working with new technology, which company officials say makes it more attractive to customers. Chief Technology Officer Peter Kirwan worked closely with Cambridge, Mass.-based Akamai Technologies Inc. to integrate graphics-caching with NaviSite's service. He says that as Web applications become more complex - requiring separate graphics and ad servers, streaming media and localized versions, for example - companies will have no choice but to rely on a partner.

For example, Kirwan heads NaviSite's streaming media service, which customers can rent. NaviSite can arrange for satellite transmission of live events over the Internet. "Sometimes (customers) have no idea how complex what they ask for really is," explains Kirwan. "And that's the beauty of the managed services model. The partner shoulders the complexity."

Herein lies one of the thornier problems in the whole ASP model: the service-level agreement. NaviSite will repair problems in applications that are based on software it knows. By certifying and mentoring its staff, it develops deep expertise in commonly used products such as Oracle databases.

Jay Seaton, NaviSite's vice president of marketing, explains that the company and its customers hammer out the service-level details up front. "A year ago, everyone promised 99.9-something percent availability; it all seemed the same," he says. "If a piece of hardware goes down, we can't possibly replace it in five minutes."

And 99.99% reliability means just that - only five minutes of downtime per month. Forty-five minutes of downtime are allowed with 99.95% availability. NaviSite is wary of making unrealistic promises in the thoroughly unpredictable environment that is the Internet, Seaton says.

The decision to use an ASP boils down to how well it can take care of an application when problems arise. To determine that, you have to visit an ASP's network operating center.

NaviSite's looks like a miniature NASA control station. Within a curved panel are computer monitors manned by the first level of defense in the troubleshooting process. At least three engineers are in the center at all times. They face five large-screen computer panels, but the most important one is on the far left. There, San Francisco-based Micromuse Inc.'s Netcool signals which problems need to be resolved. To the right, Hewlett-Packard Co.'s OpenView provides a picture of the network and its links.

The third screen runs Houston-based BMC Software Inc.'s Patrol, which enables clients to control and modify the site using remote tools. Ninety percent of NaviSite's customers take advantage of these performance management services, and the majority never come to the data center.

On the second floor of the NaviSite building, people plan for the next wave of Internet technology. The only hints of color there - as is the case within the data center and the control center - are the red tanks containing inert gases used to fight electrical fires.

The clearest lesson to be learned from a visit to NaviSite is that contingency planning - even more so than cutting-edge expertise - is the key to an ASP's survival.

Shand is a freelance writer based in Somerville, Mass.


Copyright © 2000 IDG Communications, Inc.

Shop Tech Products at Amazon