Review: Cisco's Unified Computing System wows

1 2 Page 2
Page 2 of 2

In any event, you can create a service profile that defines what firmware a blade should run on each component; what WWNN, WWPN, and MAC addresses to assign to the various ports on the blade; what management IP address to assign to the BMC; what order to boot the blade; and where the blade boots from -- local disk or SAN LUN. You can then assign that profile to either a specific blade, or you can put all the identical blades into a pool and assign the profile to the pool, letting UCS pick the blades. Here, a curious thing happens.

PXE this

Each blade is but an empty vessel before UCS gets its hands on it. With each server profile, a blade must conform to any number of specific requirements, from the firmware revision on up. Cisco accomplishes the transformation from blank slate to fully configured blade by PXE booting the blade with some network PXE magic and pushing a Linux-based configuration agent. The agent then accesses all the various components, flashes the firmware, assigns the various addresses, and makes the blade conform to the service profile. This takes a minute or two, all told. Following that, the blade reboots and is ready to accept an operating system.

This process presents a bit of a quandary: What if I want to PXE boot the OS? Through a bit of magic, the UCS configurator PXE framework will not interfere with normal PXE operations. It's apparently smart enough to get out of the way once the blade has been imprinted with the service profile. From that point on, you can install an OS as normal -- say, VMware ESX Server, RHEL 5.3, or what have you.

You can also use the virtual media facilities present in the remote KVM feature. This is somewhat old hat by now, but you can select an ISO image from your local system to present to the blade as a connected CD or DVD, and boot from that to install the OS. Here's where another funny thing happens: Generally speaking, there are no drivers to install. Windows Server 2008, RHEL 5.3 and later, and VMware ESX 3.5 U4 already have all the required UCS drivers present in the default install. You might think that Cisco's been planning this for some time. You might also think that Cisco has some significant pull with various OS vendors. You might be right.

Bouncing around the room

So you have your blades built with Windows Server 2008, VMware ESX, RHEL 5.3, or whatever. Each of them can play on however many VLANs you've defined, bind to whatever SAN LUNs you've presented, and are basically fat, dumb, and happy. So what happens when a blade goes down?

There isn't a truly defined high-availability aspect to UCS, which is somewhat disappointing. However, if you assign the server instance to a pool of blades, and it boots from a SAN LUN, then the failure of the blade running that instance will result in the instance being booted from another, identical blade in that pool. This process takes several minutes, due to the fact that UCS needs to prepare the target blade with all the specifics of the service profile, then reboot, but it does provide basic HA capabilities. It would be nice to see some form of "real" HA defined on UCS, though this poor-man's HA is functional.

Another significant facet of UCS is the concept of organizations. Cisco's management framework for UCS is not unlike that of LDAP in that it leverages the concepts of inheritance. Thus, it's possible to create organizations that have their own policies, pools, and service profiles, while child organizations can draw from the parent organization pools and so forth, inheriting policies and pools from above. This makes management simpler by allowing you to create global pools and policies that can become catch-alls, while getting more granular with those that are applied to the specific organization.

Further, administration can be delegated along organizational lines. Using another facility dubbed Locales, administrative users can be granted specific rights to specific management duties to specific organizations, with those rights flowing downhill to sub-organizations.

The tale of the scale

As with all IT infrastructure initiatives, scalability is key. Surprisingly, this isn't really an issue with UCS. Each UCS 6120XP FI can handle 144 blades with dual LAN uplinks, and the soon-to-be-released 6140s will handle up to 304 blades in the same fashion. This controller-to-blade ratio is off the charts, allowing UCS installations to scale dramatically, while requiring only the relatively cheap chassis and blades rather than the pricier FIs.

There are also significant provisions for multitenancy. For instance, perhaps you have separate working groups or even customers that need dedicated physical separation from not only each other, but through completely separate LANs. This can be achieved through the use of Pin Groups, which essentially pin specific physical interfaces to groups of servers. These can be applied on either LAN or SAN connections, so you can pin specific SANs to specific service profiles -- not specific blades.

This permits situations such as the following: Say four blades are deployed from a single service profile created for a specific department with its own LAN and SAN. These service profiles would be pinned to specific uplink ports run to that LAN and SAN. Should a blade fail, the service profile that was assigned to that blade will be brought up on another blade -- perhaps within another chassis -- yet that server instance will still maintain the physical separation as part of the pin group. This is a huge benefit for service providers and enterprises that have physically disparate network and storage segments. It places the UCS solution in the middle of any number of different network topologies while retaining physical separation, and it happens automatically.

The real tale of scalability rests with the fact that the chassis themselves are just sheet metal, a backplane, and some fabric ports. There are no smarts in the chassis, which makes them cheap. That coupled with the significant scaling of the FIs means that the more chassis you add, the cheaper the solution becomes. If there's a single lesson to take away from UCS, it's that the chassis are nothing more than extensions of the FIs, and they have more than enough bandwidth to run whatever you need. That said, once you've filled up a pair of FIs, you have to start over with a new cluster; different UCS clusters cannot intermingle under a single management domain as of yet.

Caveat emptor

To be frank, the features, scope, and breadth of the UCS offering is quite impressive for a 1.0 release. That's not to say there aren't problems. For one thing, it's not terribly clear when changes made to service profiles will cause a blade to reboot. In some instances, warnings are issued when configuration changes may cause a blade to reboot, but otherwise the state of a blade is somewhat opaque.

I encountered a few minor GUI problems and one more significant glitch: During one service profile push, the PXE blade prep boot didn't happen. A manual reboot of the blade through the KVM console got everything back on the right track, however. Throughout all the buildups and teardowns of the blades, this was the only time that happened.

Of some concern is the fault monitoring aspects of UCS. For instance, when a drive was pulled from a RAID 1 array on a running host, the event failed to throw a fault showing that the drive had failed. However, it did produce a notification that the server was now in violation of the assigned profile because it only had one disk. Further, re-inserting the disk cleared the profile violation, but produced no indication of the RAID set rebuild status. Indeed, there doesn't seem to be a way to get that information anywhere aside from a reboot and entry into the RAID controller BIOS, which is somewhat troubling. Cisco has filed a bug related to this problem and expects it to be fixed in an upcoming release.

A minor consideration is that, while Cisco is agnostic as to the make of the FC SAN attached to UCS, it must support NPIV (N_Port ID Virtualization). Most modern FC SANs shouldn't have a problem with this, but it is an absolute requirement.

Finally, there's the matter of cost. In keeping with all things Cisco, UCS isn't terribly cheap. Unless you're planning on deploying at least three chassis, it may not be worth it. The reason for this is that the chassis are relatively affordable, but the FIs and associated licenses are not. However, the scalability inherent in the UCS design means that you can fit a whole lot of blades on two FIs, so as you expand with chassis and blades, the investment comes back in spades. A well-equipped redundant UCS configuration with 32 dual-CPU Nehalem E5540-based blades with local SAS drives and 48GB of RAM each costs roughly $338,000. But adding another fully equipped chassis costs only $78,000, nearly half the price of a traditional blade chassis with similar specs.

I certainly found some problems with UCS, but they float well above the foundation, which is equally impressive for its manageability, scalability, and relative simplicity. There's a whole lot to like about UCS, and the statement it makes just might cause that revolution.

Bottom Line

Cisco UCS 1.0 is like no other blade-based server infrastructure available today. Its reliance on 10Gb Ethernet grants it plenty of bandwidth, while Cisco's model of treating chassis as simple extensions of the fabric allows for a new order of scalability and significant reliability. Cisco started from the ground up and really has built a new way to manage server resources.

This story was originally published at Follow the latest development in Cisco's Unified Computing System, blade servers, hardware, and virtualization at

Paul Venezia is senior contributing editor of the InfoWorld Test Center and writes The Deep End blog.

This story, "Review: Cisco's Unified Computing System wows" was originally published by InfoWorld.

Copyright © 2009 IDG Communications, Inc.

1 2 Page 2
Page 2 of 2
7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon