Network considerations when applying data replication technology

Protecting essential data has never been more important than it is today due to the escalating cost of system downtime and the increased likelihood of natural and human-made disasters. As a result, data replication technology is frequently being incorporated into enterprise disaster recovery plans. Determining how to best apply this technology is an important step in successful planning. There are a number of factors for IT managers to consider, including data replication mode of operation, networking options and geographical resiliency.

With data replication technologies, valuable business data is duplicated, or "mirrored," from one highly available storage disk array to another. The movement of this data occurs in real time, typically over a high-bandwidth, dedicated network (see Figure 1). Data replication can occur in a synchronous, asynchronous or semisynchronous mode of operation. In synchronous mode, block-level data transactions or I/O requests are committed to one or more remote storage systems prior to sending a write acknowledgment back to the primary unit.


Figure 1: Network Storage Environment

The host application isn't allowed to proceed until data is successfully committed to all storage systems. For this reason, synchronous mode is the most sensitive to network latency and bandwidth.

In an asynchronous mode of operation, I/Os are written to the primary storage system and then sent to one or more remote storage systems at some later point in time. The host application functions as usual, unaware as to whether or not the data has been written to the remote storage systems. The network interconnecting the storage systems doesn't affect this mode of operation as significantly as it does the synchronous mode.

As the name implies, semisynchronous is a hybrid approach, whereby I/Os are written on the primary storage system, the application is allowed to proceed, and the data is meanwhile synchronized with the remote storage systems. This approach is similar to asynchronous with the exception that attempts to synchronize data on remote storage systems occur immediately (minimal time lag). Therefore, the network performance is once again an important consideration.

It is common for a company to incorporate more than one method of data replication to protect its valuable business data. The mode of operation will depend on the business value of the data, the network used to secure the data and the distance between storage systems.

Networking options

The main reason for including data replication in disaster recovery plans is to protect mission-critical data. Using multiple storage systems helps prevent issues associated with equipment failure through hardware redundancy. However, it's also extremely important to secure data in multiple geographical locations to minimize the potential for data loss due to natural or human-made disasters. Therefore, a good disaster recovery plan typically includes geographically dispersed storage systems. As mentioned above, data replication (especially via synchronous and semisynchronous modes) requires minimal network latency and high bandwidth. With these sensitivities in mind, storage-networking options for interconnecting multiple data centers include:

  1. Coarse or Dense Wavelength Division Multiplexing (C/DWDM)

  2. Synchronous Optical Network (Sonet) or Synchronous Digital Hierarchy (SDH)

  3. Storage over IP

C/DWDM puts data from different sources and protocols together on an optical fiber, with each signal carried on its own separate and private light wavelength. Using C/DWDM technology, up to 80 (and, theoretically, even more) separate wavelengths carrying data can be multiplexed into a light stream transmitted on a single optical fiber, providing a cost-efficient utilization of an enterprise's network resources. C/DWDM is well suited for natively (no protocol conversion) extending storage-area networks (SAN) in cases where multiple communication links or channels are required. It has been verified to support data replication over distances up to several hundred kilometers (see Figure 2).


Figure 2: Natively Extending SANs by C/DWDM

Sonet/SDH technology is based on time division multiplexing, where traffic from multiple subscribers is multiplexed together and sent out onto the network as an optical signal. Sonet/SDH services are widely available within metropolitan areas around the world. Sonet is the North American standard; SDH is used elsewhere. New technologies such as generic framing procedure (GFP) and virtual concatenation (VCAT) enable multiple protocols (including storage protocols) to run over Sonet/SDH links. GFP and VCAT map SAN traffic into a virtually concatenated Sonet/SDH pipe. The combination of Sonet/SDH, GFP and VCAT enables optical connectivity that can scale from 50Mbit/sec. to 10Gbit/sec., meeting the needs and budgets of a wide range of IT environments. With this technology, enterprise data centers can be interconnected over thousands of kilometers for data replication and other storage networking needs.

René Dufrene

Storage over IP enables transportation of block-level storage over an IP network via protocols like iSCSI, FCIP (Fibre Channel over IP) or iFCP (Internet Fibre Channel Protocol). Data rates are usually near or less than 155Mbit/sec. Storage over IP enables storage traffic to extend great distances via a variety of wide-area networking options such as T1/E1 and T3/E3 circuits, Asynchronous Transfer Mode, frame relay and Ethernet. Synchronous or semisynchronous mirroring may be difficult because of the higher, less predictable latencies associated with IP networks.

Network design considerations

Characteristics of an inter-data-center storage network infrastructure to support data replication generally include:

  1. Low latency

  2. Minimal packet loss

  3. Flow control mechanisms

  4. Network resiliency

  5. Scalable bandwidth

Data replication, particularly in synchronous mode, requires in-order packet delivery, latency tolerances in the handful of millisecond (one-thousandth of a second) range, data rates per communication link near 1G or 2Gbit/sec., and network path redundancy. Deterministic optical networks can provide all of these most effectively. An optical network is capable of delivering data 200 kilometers in 1 millisecond, or from New York to Los Angeles in 20 milliseconds. As discussed above, optical networks are very scalable. Restoration times in the event of a fiber cut on a transport path are under 50 milliseconds. Unlike an IP-based network, Sonet/SDH and C/DWDM are designed for loss-less data transmission, which is an important factor in a data replication environment. Optical networking equipment now provides buffer credits and caching mechanisms for flow control in SAN environments to enable efficient use of the network link.


IT managers must select from a variety of options when using data replication as part of a company's disaster recovery plan. The mode of operation will depend on the data protection requirements and network characteristics. Any data replication plan should include geographically dispersed storage systems. As indicated above, optical networks are ideal for supporting data replication. Sonet/SDH is scalable, reliable and very affordable; meanwhile, C/DWDM is well suited for business continuity/data recovery infrastructures with moderate-to-high inter-data-center bandwidth requirements. Careful analysis of flow control, tolerances to packet loss and latency, bandwidth requirements, and network resiliency is needed to successfully implement a sound data replication strategy in the enterprise.

Copyright © 2003 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon