North Bronx Health Network finds the cure for data overload

North Bronx Health Network (NBHN) has long been a pioneer in electronic medical records systems. In fact, it was in the early 1990s that Jacobi Medical Center, a hospital that later became part of the public urban hospital network, piloted one of the nation's first electronic health record systems.

But over time, all that data - which grew to 100TB, dispersed on 75 different clinical information systems, each with its own direct-attached storage system - became increasingly difficult to manage. About 20 months ago, Daniel Morreale, NBHN's CIO, decided this storage approach "just didn't cut it anymore."

"Everything happens online," says Morreale. Medical staff use PCs and portable computing devices to enter or update patient records from anywhere in a facility. Doctors' reports, physician orders, admission notes, discharge summaries, notes on patient consults, nurses' notes - all of these are entered into a computer system. X-rays and all other images reside on a Picture Archiving and Communications Systems server.

All this electronic data has enabled NBHN to dramatically reduce the incidence of medication errors, cut response time in emergencies and save millions of dollars. However, it has also generated a huge amount of data, which the IT staff was having increasing trouble managing, Morreale says. "We had 200GB available here, 75GB there. We couldn't manage or maneuver the data or fully capitalize on our available storage resources," he says.

Adding urgency to NBHN's need for a more effective storage infrastructure are recently instituted government regulations like HIPAA that require the hospital not only to guard patient data and make sure it's confidential but also to maintain it in usable format for between seven and 25 years, Morreale says.

A lifetime of data

Working with EMC Corp., Morreale and his staff came up with a multi-tiered, hierarchical, networked storage infrastructure that will eventually support policy-based information life cycle management (ILM).

"The whole concept of lifetime data management is new to health care and will need time to mature," Morreale notes. However, "ILM is absolutely strategic to the success of our electronic health-care records infrastructure. It will enable us to ensure that our medical and administrative staff have secure and rapid access to patient data from the time it's created until it can be disposed of - often decades later."

Now in the final stages of implementation, NBHN's new 230TB infrastructure consists of the following:

  • EMC Symmetrix DMX: The SAN-based, high-end platform houses patient-critical data and supports high-transaction applications. The Symmetrix system provides pooled, virtualized storage for a range of servers running a variety of operating systems, including IBM AIX, Microsoft Windows NT, Data General Unix and Sun Solaris.

  • EMC's ControlCenter: This system allows a storage administrator to monitor utilization, as well as to anticipate and meet the storage needs of new and existing applications.

  • EMC Celerra NS600: This NAS supports file sharing for applications that don't need the Symmetrix SAN's high speeds, for example, if a doctor wants to update a patient record with the latest diagnosis or treatment.

  • EMC Centera: This low-cost, content-addressed storage (CAS) platform houses older, less critical data.

  • EMC's mid-tier Clariion CX600:This ATA storage system provides centralized backup and recovery of files.

NBHN has already realized significant returns from the new infrastructure, including a 50% reduction in its server population and an increase in storage capacity utilization from 40% to 70%, Morreale notes. His group has realized significant savings from not having to buy storage for each server, which added up to about $400,000 for a recent acquisition of IBM 6250s.

By consolidating storage resources onto a centrally managed architecture, "we've made it reasonable for one person to manage storage on a daily basis," Morreale says. EMC ControlCenter has also simplified automated path provisioning, load balancing and path fail-over on the Symmetrix SAN, as well as volume management.

The beginnings of ILM

Morreale's group is in the early stages of implementing ILM on the new storage architecture. It is implementing policies as scripts that tell EMC's ControlCenter where a patient record or image should reside over its lifetime. For example, a newly created patient record might reside on the Symmetrix, where it can be accessed quickly for six months and then move to the Centera's long-term archive.

NBHN chose the Centera CAS platform as an inexpensive medium that provides access to archived material in seconds - much more quickly and reliably than a tape drive, Morreale says. The CAS platform now holds about 120TB of archived data, including patient records, images and some electronic messages. The organization is using Microsoft Exchange 2000's e-mail archiving feature in order to meet a HIPAA requirement that messages containing patient data be retained over time.

Another useful CAS feature is the ability to store and call up records as objects, each with its own digital fingerprint. This provides legally valid assurance that the record has been retained in unaltered fashion, meaning that an unauthorized person didn't call it up, change it, then store it under the same name, for example, says Roy Sanford, EMC's vice president of content-addressed storage. "The legal term is that the record has evidentiary weight," he says.

The ability to produce a record that is intact and provably unaltered can be crucial, Morreale points out. "If anyone contests treatment or has an issue, we can go back and validate the episode of care."

Furthermore, medical staff in NBHN's two hospitals must be able to access years-old patient records quickly and reliably to provide quality of care over time, Morreale notes. When a patient shows up in the emergency room after a hiatus of several years, for example, hospital staff can look up the treatment history and access X-rays on the CAS platform in seconds.

Yet another reason why NBHN chose CAS: The health-care organization is moving toward a 100% lights-out computer shop, requiring automated and remote management capabilities that tape drives cannot support.

Security and business continuity

The new infrastructure, with its built-in redundancy and automated management software, also enables NHBN to address security and business continuity requirements - both the government's and its own - without breaking the IT budget.

EMC Data Manager provides backup in a computer room in one hospital and also backs up to an off-site facility. EMC's SRDF system provides remote replication. Currently, NBHN has enough storage space on its SRDF site to back up a year's worth of data. Morreale plans to expand redundant capacity to back up a full seven years of data on primary systems.

Security has become much less problematic and people-intensive now that data resources have been consolidated and centralized, Morreale indicates. Storage devices and servers are now housed in a controlled computer room with redundant power supplies. Access requires handprint authorization and a password. In addition, every electronic device is locked down in a cabinet with a key. End users can only access data through applications that need password validation on Microsoft's Active Directory.

The new infrastructure will enable NBHN not only to comply with current regulations but also to react dynamically to regulatory changes over the years, by assigning storage to new systems with minimal preparation and cost, Morreale says. "I know where my data is, I know it's secure and backed up, and I can move it across various storage levels as it ages," he says. "We now have the ability to make lifetime data management a reality."

Elisabeth Horwitt is a freelance writer based in Waban, Mass.

Copyright © 2004 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
 
Shop Tech Products at Amazon