Skip the navigation

Upstate New York's supercomputing may cause envy

By Jim Duffy
January 9, 2007 12:00 PM ET

Network World - Editor's note: This story contained some inaccuracies in the 26th and 27th paragraphs; it was corrected on Monday, Jan. 22, around 11:55 a.m.

The 100-acre medical campus in downtown Buffalo is considered by many here to be the rose amid the rubble of this rusty industrial city still recovering from an economic collapse that took place 30 years ago. The Buffalo Niagara Medical Center, also called "the corridor," employs 8,000 people among four main health care and research institutions and has a supercomputing center that could be the envy of the Northeast.

The Center for Computational Research (CCR) at the State University of New York at Buffalo operates a 1,200-server system that supports about 100 research groups spanning nearly 40 departments at the university, as well as more than 20 companies and institutions in the Buffalo-Niagara region. Among them are the world-renowned Roswell Park Cancer Institute, the Hauptman-Woodward Research Institute and the new New York State Center for Excellence in Bioinformatics & Life Sciences, where the system is housed.

The CCR system also serves as the hub of a statewide grid connecting 18 educational and research institutions. But in its primary role as the computational heart of the medical campus, CCR's system can analyze the complex structure of cells and proteins in days vs. months or even a year when using a series of workstations, says Director Thomas Furlani.

In addition to bioinformatics, which includes DNA and protein sequencing, gene expression and biological pathway analysis, CCR's system supports computational chemistry, data mining and database development, earthquake engineering, environmental modeling and simulation, grid computing, homeland security, visualization and urban planning, and virtual reality and animation.

On the homeland security front, the Jacksonville, Fla., city school district is having CCR construct a prototype visualization of its schools so if there's a shooting, for example, law enforcement and school officials can virtually look in every nook and cranny for the assailants or available exits.

"You never know where this is going to lead," Furlani says.

CCR's system consists of parallel processing servers connected by 2Gbit/sec. bidirectional links, with a 100BaseT connection for maintenance and administration. It also includes sequential processing servers linked via 1Gbit/sec. Ethernet.

The largest cluster in the group includes 1,056 Dell PowerEdge SC1425 servers, each of which has two 3-GHz or 3.2-GHz Xeon processors and varying amounts of main memory: 964 nodes configured with 2GB; 64 nodes configured with 4GB; and 32 nodes configured with 8GB. The servers are interconnected with a pair of Force10 Networks Terascale E1200 Gigabit Ethernet switches and run the Red Hat Linux operating system.

The peak performance of this cluster is just over 13 teraflops -- or 13 trillion floating-point operations per second -- and all machines run at 85% to 90% of capacity, Furlani says. The University at Buffalo acquired the bulk of these servers in 2005 and the remainder in 2006.

CCR also employs a 35TB EMC Corp. storage-area network with I/O performance of 3GB/sec. In all, CCR has more than 300TB of storage, and combining this with the 13 teraflops of compute power makes it No. 87 on the list of the top 100 supercomputing centers in the world, according to supercomputing site tracker

That's why CCR is also the hub for a regional grid that connects 18 other institutions across New York. Called NYSGrid, the network aggregates high-end computing, networking, data storage, visualization and intellectual capital from sites across the state to provide its constituency with resources for research, education and community outreach.

NYSGrid also fosters development of software, algorithms, portals and interfaces that will let researchers and students collect, manage, organize, analyze and visualize data without having to worry about details such as where the data is stored, processed or rendered.

Another participant in NYSGrid -- and a client of CCR's -- is the Hauptman-Woodward Research Institute (HWI), located directly across the street. CCR acts as a service provider to HWI, delivering the computer cycles and raw bandwidth required to conduct protein crystallization research and imaging for more than 700 institutions, says Dan Degnan, HWI's director of IT. "We produce camera image files that are sent back to scientists showing what chemical compounds they can use to crystallize proteins," Degnan says. "They can reverse-engineer the structure out of this."

HWI handles 200 proteins a month, Degnan says. Its own network is an 88-node Apple Computer Inc. cluster connected with 1Gbit/sec. Ethernet links to Cisco Catalyst 2600, 3560, 3750 and 4507 switches.

HWI is connected to CCR by means of OC-3 links.

"We're world known for our high-throughput lab," Degnan says. "We get [research] submissions from all over the globe."

Though CCR's focus is predominantly on life sciences, many of its compute cycles are expended on commercial and civic endeavors. Hockey fans may not know that the animated imagery before and during games broadcast on the Versus cable TV network is rendered at the CCR on behalf of local animation company IBC Digital. The same is true for IBC music videos shown on MTV2.

"It would have taken IBC Digital six months per video, but their entire contract was six months, and they had 32 videos to do," Furlani says. "So we did each video in six hours."

CCR also provides the compute power behind research at the University at Buffalo's School of Management into the buying habits of customers at the Tops Supermarket chain. CCR is housing 4 billion records associated with the Tops market research in its database, Furlani says.

Also on a local scale, CCR is crunching out the visualization for civic projects such as returning vehicle traffic to a downtown section of Buffalo's Main Street, which was made into a pedestrian mall in the 1980s as a new light-rail system emerged; building a new bridge across the Niagara River to Canada; relocating a New York State Thruway toll plaza farther out into the suburbs to relieve congestion coming into the city; assessing current and future plans for several Buffalo city parks through the Buffalo Olmstead Parks Conservancy Restoration project; and modeling the university's plans for expansion.

Half of the visualization projects undertaken at CCR are for economic development projects, Furlani says. The University at Buffalo did $200,000 worth of contract visualization work on behalf of local companies and organizations last year, he says.

The Buffalo Niagara Medical Campus overall contributes $300 million annually into the local economy.

"It's not just research computing in the life sciences," Furlani says. "It's high-profile community involvement, and people understand it. It presents a lot of potential for moving projects forward."

In that respect, CCR could play a key role in turning around Buffalo's fortunes. It could help bring people back to a city that has lost more than half its population in the past 50 years.

Furlani believes the raw compute power located in the medical campus corridor could attract life sciences companies to set up their headquarters in Buffalo. It has also put CCR in the running for a $200 million National Science Foundation grant to help one of the country's largest supercomputing centers construct a petascale computer by 2010 to be used by researchers across the country.

The grant will be awarded later this year.

"We're helping the university get more research grants. That's our mission," Furlani says. "But there's an economic development component -- if we can make local corporations more competitive, we're all for it. If we can help train people to go out and get jobs at these firms, that's great."

Reprinted with permission from Story copyright 2012 Network World, Inc. All rights reserved.
Our Commenting Policies