Executives at online auction provider FreeMarkets Inc. had a problem. Explosive growth was creating demand for higher scalability and uptime from the data center, but the Windows NT Server 4 infrastructure -- a Web server farm with back-end systems built on top of Microsoft's SQL Server 7 -- was running out of gas. With IT systems at the core of FreeMarkets' business, it was a problem that could strangle growth.
The Pittsburgh-based company helps business users set up and manage online auctions for industrial parts and other commodities using FreeMarkets' FullSource and DirectSource services and FreeMarkets Desktop end-user application. Sales have grown fast. In the first half of last year alone, the company exceeded the $3 billion in trades it had executed the previous year.
To support that growth, says Tony Bernard, FreeMarkets' director of technical architecture, the company needed "five nines" availability (99.999% uptime) and a more scalable infrastructure than its NT-based systems offered. "We would reboot our servers once a month," he says.
FreeMarkets also wanted to implement QuickSource, a major upgrade of its do-it-yourself auction service. The company had to pull this off without affecting the availability of its applications.
A Matter of Choice
In late 1999, FreeMarkets made the initial decision to migrate the data center to the yet-to-be-released Windows 2000 operating system and to overhaul the company's production servers "to have a more scalable and reliable architecture for our application environment," recalls John Benzinger, FreeMarkets' vice president of IT.
Windows 2000 was promising the scalability and reliability FreeMarkets needed. But it wasn't a shoe-in: With a large number of its key applications written in Java, the company seriously considered using Linux. "The drawback was that in most cases, it was going to mean a forklift upgrade for our data center and complete retraining," Bernard says. The company determined that going exclusively with Microsoft Corp. development tools and architecture would help simplify product development in ways not possible with a more Java-centric approach, he says. "There were too many vendors, too many components, too many pieces of the puzzle," Bernard says of Java.
By early last year, a plan was put in place to migrate Web servers and QuickSource database servers in a "guinea pig" phase for the migration project.
FreeMarkets programmers were confident that they could handle the Web server upgrades, says Bernard. But as the company began a redesign of its database tier, it called in Brenda Basista, a consultant in the Enterprise Consulting Services division at Compaq Computer Corp., to do a high-availability assessment of the existing environment.
"They had a lot of redundant servers," Basista recalls, with key databases scattered across eight Windows NT 4 machines. After three months of analysis, "we determined that clustering was the right way to go," she says. "We started playing with Microsoft [Windows 2000] Advanced Server in a clustering environment."
By early spring, a team of Benzinger's operations experts had tested some auction components on Windows 2000 servers, and Mark Tyrrell, a consultant at Microsoft's Pittsburgh office, joined the effort..
In July, FreeMarkets entered Microsoft's Joint Development Program (JDP) for Windows 2000 Datacenter Server. "We thought we could learn a lot about Windows 2000 by participating in that Datacenter JDP program," says Bill Blair, FreeMarkets' chief technology officer.
FreeMarkets also ran a beta version of QuickSource on Windows 2000 servers that it had set up in a test lab. "Quite honestly, there was a litmus test," says Blair.
"We kind of knew how that application performed on NT 4," Bernard recalls. "We ran it on the new environment, and we saw a noticeable improvement."
If You Build It ...
The team created an architecture that includes a QuickSource database application server cluster (dual Compaq ProLiant 8500 eight-CPU servers with 16GB of RAM running Datacenter Server and SQL Server 2000) with an enterprise application server cluster (dual ProLiant 8500 servers running Advanced Server) that handle interactions with suppliers. A Compaq StorageWorks storage-area network (SAN) holds the databases. The SAN and the servers are linked via Gigabit Ethernet to dual-clustered back-office application servers running Datacenter Server and OneWorld enterprise resource planning (ERP) software from Denver-based J.D. Edwards & Co., which handles accounting and billing. A nonclustered Web server farm running Advanced Server serves QuickSource pages and handles incoming requests using LocalDirector load- balancing hardware and software from San Jose-based Cisco Systems Inc.
Only clustering could provide the needed fail-over features for the dynamic databases, Bernard says. FreeMarkets chose Datacenter Server because it scales to a four-node cluster and supports more memory and processors than Advanced Server. But the less-expensive combination of Advanced Server and LocalDirector was the best design for the Web servers.
"It's much more economical and efficient and flexible to do load balancing," Bernard says. "Our Web tier is typically where the bottleneck is, in terms of performance. [Datacenter Server] is just a more expensive configuration." He says that this setup makes it easier to scale front-end capacity by swapping out older servers and replacing them with faster ones.
Last September, FreeMarkets rolled out its QuickSource beta (then called Self Service), the first production application, on its new architecture. Some of the company's databases and basic auction components had also been moved onto the Windows 2000 servers, according to Benzinger. "We have a lot of supplier information that is part of all of our products," he says. "Those databases were migrated over." Bernard says he was satisfied. "We didn't have any issues I would even classify as significant," he says.
Uptime across the new system improved noticeably, Benzinger says. Monthly reboots, common with NT 4, became a thing of the past. But each reboot took 30 minutes longer when it was needed, he adds. Performance also improved; Web pages, for example, seem to load in about half the time, Bernard says. The new multiprocessor Compaq servers account for part of that difference, but "Windows 2000 itself is a faster operating system," Bernard says, and "SQL Server 2000 seems to perform better."
Both Bernard and Benzinger say they're impressed with the new system's reliability. The only negative, they say, has been the unavailability of Windows 2000 software drivers for running hardware such as laser printers from Hewlett-Packard Co. Also, porting custom applications from NT 4 to Windows 2000 has been tricky because the names of software libraries have changed. Benzinger and Bernard also say they fault Microsoft for having a licensing scheme that overcharges for copies of Windows 2000 that are needed for low-priority uses. "I don't need five-nines on my development server," Bernard says.
FreeMarkets' main challenge now is coordinating the various upgrades on the new servers, including the migration of its DirectSource and FullSource applications. Also, the company wants to add new e-business customer service, sales and marketing software from San Mateo, Calif.-based Siebel Systems Inc., but the certified upgrade won't be available until early spring. "We are not pleased that we don't have a 2000 version of that now," Blair says.
Earlier this month, just as QuickSource was about to ship, Basista was preparing for a major milestone: porting the Bidware bidding engine, used in FullSource and DirectSource, from SQL Server 7. The upgrade will later be certified in a Compaq laboratory. Unfortunately, the Bidware upgrade could be ready for months before Compaq can schedule it for certification, likely around midyear.
Also earlier this month, the company was preparing to go live with a partial migration to Windows 2000's Active Directory (AD) service, which Benzinger says will facilitate directory management and allow for consolidation of domain controllers. The AD rollout, however, creates quality-assurance issues that must be addressed with extensive retesting that has so far taken two months, he says. But FreeMarkets remains focused on the ongoing data center migration. "Our goal is to build a highly available Datacenter Server cluster running SQL Server 2000 and consolidate all of our databases," Bernard says.
So far, the results have been promising. FreeMarkets says maintenance costs have declined by 20% and auction capacity has risen 70% since the migration began in late 1999.