High-speed Ethernet planning guide

Everything a network architect needs to know about migrating to 40/100Gigabit Ethernet

1 2 Page 2
Page 2 of 2

Bulking up with fiber

Unlike 10G Ethernet, where fiber-optic cables use only two fiber strands, optical cabling for 40G Ethernet currently uses 12 fibers: Two groups of four fibers send traffic in each direction, and four fibers between the two groups are left dark.

Multi-lane 100G Ethernet requires even denser optical cabling. Here, cables use 24 fibers: Two arrays of 10 fibers for traffic in each direction, with two fibers on each side of each array left dark.

Both 40G- and 100G flavors of HSE require "laser-optimized" fiber cables. For multimode fiber, this means using OM3-type cables for distances of up to 100 meters. For longer spans, OM4-type cables support runs of up to 125 meters.

OM3 and OM4 cables (which usually have aqua-colored jackets, to distinguish them from older OM1 and OM2 cables with orange jackets) are already recommended for 10-gigabit Ethernet, but that doesn't mean OM3 and OM4 cables can be used interchangeably between 10G- and 40G interfaces.

HSE will require new cabling, but Torrey Point's Ross notes that a migration could end up reducing the numbers of interfaces and cables needed. "People today use multiple, aggregated 10-gigabit connections" for network uplinks, Ross says. "When they move to 100-gigabit Ethernet, they actually reduce fiber demand."

There are two significant differences when it comes to HSE optical cabling: Fiber count and transceiver types. As noted, fiber optic cabling for multi-lane HSE uses either 12 or 24 fibers, compared with two fibers with 10G Ethernet. Thus, network planners will need to specify whether 10G-, 40G-, or 100G versions of OM3 or OM4 cabling are needed.

The transceiver most commonly used for 40G Ethernet is called quad small form-factor pluggable plus (QSFP+). This connector, available for both fiber and copper cabling, resembles a slightly larger version of the SFP+ connectors used in 10G Ethernet, with the "quad" designation referring to its four pairs of transmit and receive lanes. Although QSFP+ transceivers only support 40G Ethernet today, they may support 100G Ethernet in the future when lane rates increase.

For now, 100G interfaces use C form-factor pluggable (CFP) transceivers. These are slightly larger than a smartphone and support single-mode optical cabling with 24 fibers. CFP transceivers also can support multimode fiber and copper cabling, though QSFP+ is more common. A smaller variation of CFP, called CXP, will support all the various HSE versions but these are not yet widely available.

Requirements for copper cabling are somewhat simpler than those for fiber. Like fiber, HSE copper cabling uses multiple transmit and receive channels, each carrying one lane running at 10-gigabit rates.

Copper interfaces are defined for both 40G- and 100G versions of HSE, but only for short-reach applications such as connections within a rack or between adjacent racks in a data center. Using the currently defined interfaces, HSE connections over copper can be at most 7 meters apart.

QSFP+ transceivers are used for 40G Ethernet over copper. Some large enterprise data centers already use 10/40-gigabit top-of-rack switches. These typically support 48 SFP+ ports for 10-gigabit connections to servers, and four QSFP+ ports for 40-gigabit Ethernet uplinks for switch-to-switch connections.

Conversion between 10- and 40-gigabit rates is an especially useful feature of QSFP+ ports, both for copper and fiber. In many top-of-rack switches, a QSFP+ port also can function as four 10G Ethernet ports. This involves the use of copper or fiber "breakout" cable with a QSFP+ connector on one end and four SFP+ connectors on the other. Both sides see the connection as four 10G Ethernet interfaces.

This convertible feature offers a migration path for data centers planning, but not yet ready, to deploy 40G Ethernet. It's also cost-effective to buy and manage just one type of switch and then deploy either 10G- or 40G Ethernet ports as needed.

Direct attach copper (DAC) cables already are a common means of connecting servers and switches over short distances at 10-gigabit rates, and that's likely to continue with 40-gigabit DACs. The 40-gigabit units, which comprise two QSFP+ transceivers connected by a run of twinaxial cable, are typically 3 meters or less in length.

If longer spans are needed (for example, to connect racks more than 7 meters away from a core switch), network managers will need one of the fiber-based options.

In addition to QSFP+ connectors for copper or fiber, there's another fiber-only connector type known as multi-fiber push-on (MPO). These connectors support the parallel-optics approach and require MPO connectors on fiber cabling and MPO interfaces on switches and routers. The LC and SC fiber-optic connector types used in earlier versions of Ethernet aren't suitable for HSE because they terminate only two fibers.

Migrating to HSE will certainly require changes. The good news is that the changes are evolutionary in nature, and can be rolled out incrementally. Ethernet has undergone many changes over the years, but the traffic on HSE networks retains the same basic characteristics as previous generations. In the end, it really is "just Ethernet."

Newman is a member of the Network World Lab Alliance and president of Network Test, an independent test lab and engineering services consultancy. He can be reached at dnewman@networktest.com.

Read more about lan and wan in Network World's LAN & WAN section.

This story, "High-speed Ethernet planning guide" was originally published by Network World.

Copyright © 2011 IDG Communications, Inc.

1 2 Page 2
Page 2 of 2
Shop Tech Products at Amazon