Ethernet: It's not just for data networking anymore

Recently, the Internet Engineering Task Force (IETF) completed its work on an iSCSI specification that leaves the industry with a standard for building storage networks based on Ethernet/TCP/IP technology (see story). With this new specification, the industry is in a position to take advantage of the capabilities of iSCSI and find out the true impact that it can have.

Based on the number and frequency of iSCSI product announcements, it would appear that end users will increasingly be building IP-based SANs and connecting remote locations or doing data backup over distance using iSCSI routers. This article will examine the realities of such initiatives and what the new IETF specification brings to storage networking.

Before focusing on iSCSI, it is important to understand why companies build storage networks in the first place. Extensive focus group studies have shown a consistent set of drivers for data storage, including:

  • Large image files - engineering, meteorological, medical, etc.

  • Databases - they only grow

  • E-mail - increasingly used to distribute large files among groups

  • Storage buffer - while swapping out systems

  • FDA and SEC rules - need to save every e-mail and every other document, including all revisions for five to seven years

  • CAD software-still growing.

How is all this data stored now?

Outside of the largest companies that have deployed SANs, most storage today is connected via DAS (direct attached storage) devices. DAS has been the predominant method of storage expansion for a long time. When first deployed to expand storage outside the server, DAS seems like an easy way to go.

But, as the individual DAS devices start filling up, storage managers realize the downsides of DAS including: there are many separate devices to manage and upgrade, available storage isn't always attached to the server in need, there is no easy way to replace a storage device, and backing up data is cumbersome. In sum, the whole issue of data backup that has become so critical today is hard to do with the separate storage islands presented by DAS environments.

To make storage management easier, many IT managers are beginning to look to either NAS or SANs. Fundamentally, SANs handle block-level data and NAS devices are for file-level data. (See below for an explanation of the two and how they work together).

For small to mid-size companies, NAS appliances have increasingly been deployed as relief from the DAS island problem. With their ease of installation and file-level orientation, these devices have provided small storage management staffs with a quick, low-cost solution for storage expansion. The downside is that NAS is so easy to install that, as NAS appliances fill up, others tend to be quickly installed, creating islands of NAS.

There are still, however, fewer NAS islands than DAS islands. To address this issue, both start-up storage companies and large established industry players will be introducing scalable NAS appliances. Basically, these new NAS boxes can be deployed as needed, and present themselves as large file-level storage pools. The storage management software running on these devices allows auto-discovery of other NAS appliances and pooled sharing and backup of data. Call it the virtualization of NAS.

As previously mentioned, SAN solutions are commonly deployed at large companies because there are multiple SAN inhibitors at smaller firms. The most obvious obstacle is that SANs handle block-level data, which is relatively rare at small companies. Other barriers include costs and security. SANs are typically built using Fibre Channel (FC) technology. Smaller companies usually employ Ethernet data networks, and have no expertise in FC, nor the budget to hire additional staff with FC experience.

Small companies also have shied away from FC technology because until approximately three years ago, it was plagued by interoperability problems. Security in FC SANs is largely derived by physical FC network isolation from the Ethernet data network. Even though FC SANs might not be too technically challenging, they are still time-consuming for stressed-out small staffs that already have their hands full with Ethernet.

In mid-range company focus groups, it is clear that SANs would have more appeal if they could be built using the well-understood Ethernet/IP technologies already in use. One other critical point also becomes clear: IP-based SANs will not be built by buying the components piecemeal. Users expect the complete bundled-solution-sell common to the Fibre Channel arena. This means that storage devices, management software and HBAs need to be tested and sold as packages. This has not happened yet.

Today, it is possible to purchase iSCSI HBAs. These HBAs were available long before there were any iSCSI storage devices for them to assist. The big hurdle is the lack of packaged solutions. Expect to see these later in 2003.

How will iSCSI be implemented on storage devices and servers?

Given that the major storage HBA players have announced or started shipping iSCSI HBAs, it would be natural to think that HBAs are the way to go, but is that the case? To have this discussion requires that we look at the target and initiator sides separately, as the problem to be solved for each differs.

Starting with the target side, what we see today in the few iSCSI storage devices that are shipping is a software iSCSI target implementation. These are either purpose-built iSCSI storage boxes, such as the Eurologic 2100, or NAS Filers such as Network Appliance's FAS900 series that includes an optional free iSCSI software upgrade. These vendors have taken their implementations to the iSCSI plugfests and are confident in their interoperability. They will also be willing to explore the performance gains and CPU load savings available when the hardware ASIC-based HBAs are proved to be interoperable.

Targets such as Network Appliance's are basically filers and need file-level acceleration as well as the iSCSI block level in hardware. This requirement has complicated the HBA decision by putting the HBA in a position to do both jobs. Over time though, as the ASIC solutions mature and are proved stable, most target implementations will migrate to ASIC-based HBAs.

On the initiator side, there are several competing solutions available, with the lowest cost being the free software iSCSI initiator stacks from Microsoft for Windows. There are also Linux and several Unix software solutions available from Cisco and others. Many storage vendors are initially recommending the use of these software solutions as they assess the performance and CPU load issues, and get a better understanding of when HBA solutions will be required.

A lot will depend on the types of infrastructures that companies have for their IP SANs. If Gigabit Ethernet infrastructures are in place, link speeds require an ASIC-based HBA to operate at line rate without relying heavily on the host CPU. If the infrastructure is a 10/100M network, software solutions can easily support this slower link speed.

What about security?

With widespread connectivity to storage across entire enterprise networks, and the movement of storage data on the Internet, there are very valid concerns about data access and security. The iSCSI specification covers security with support for IPsec. There are four elements to IPsec: login authentication, packet authentication, data encryption and key management.

The question remains as to where security will be invoked. So far, it doesn't seem that end users will burden the devices in their data centers with full IPsec support for data that moves within the data centers. All agree, however, that data on the move outside the enterprise must be encrypted. Expect to see the deployment of gateway security boxes supporting IPsec at access points to the enterprise network.

iSCSI's biggest impact on future storage networks

With the ability to support block level data on Ethernet infrastructures, it will no longer be necessary for separate storage networks to be built for block and file support. New storage appliances that support both block and file access will make it far simpler to build a single converged infrastructure.

As a result, iSCSI HBAs will have to evolve into a multi-mode HBA devices supporting iSCSI block data as well as several types of file level acceleration. These include TCP/IP offload, TCP/IP offload with NFS/CIFS parsing/ acceleration, or RDMA (assuming the IETF moves RDMA forward and there is wide vendor support). The resultant storage infrastructure will be easier to build, manage and expand.

1pixclear.gif

SAN and NAS: are they related?

What's an IP SAN? Basically the same as a Fibre Channel SAN, but the infrastructure is comprised of Ethernet/TCP/IP components. Both handle block level data. The switches are standard Ethernet switches as used to build the company's data network. The only noteworthy feature recommendation is that they support Ethernet Jumbo Frames. This is because many applications use block sizes in the 4K to 8K byte range, and with standard Ethernet maximum frame sizes of 1518 bytes, multiple frames are required to transfer a block. Most Ethernet switches today support Jumbo Frames of 9K bytes allowing a single frame to carry a whole block, which is much more efficient.

What can an IP SAN do that a FC SAN cannot do? Since the infrastructure is pure Ethernet/TCP/IP, backup over distance and remote office connectivity become very easy using either private WAN links or the Internet. Connectivity to any Fibre Channel SAN-based data is supported by use of a storage router.

How are NAS and SAN related? NAS filers support file-level access to storage. This is a very well understood storage method by users. To provide file backup, storage expansion and companywide shared file storage, NAS appliances are widely deployed. Now, internal to the filer are disk drives, and the data is stored on those drives as blocks, just as data is stored as blocks in laptop and desktop drives. If the filer disk space required expansion, external drives can be added — either via Fibre Channel or iSCSI — thus creating a SAN behind the filer.

Given that the filer has block data storage internally, most NAS appliances are allowing for block-level access as well as file-level access at the front end. As an example, today Network Appliance ships its filers with optional FC support and iSCSI support on the front end. This is changing the view of how storage networks are built. No longer are they going to be pure block level networks, but combined block/file storage networks.

Charlie Kraus is director, marketing and product management, at LSI Logic.

Copyright © 2003 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon