Xgrid -- A first look at Apple's new cluster software

Like many of the Macintosh faithful, I watched Steve Jobs' keynote speech at MacWorld last week with anticipation, and I wasn't disappointed. The announcements about the new G5 Xserve and the beefed-up Xserve RAID were just what I had been hoping for in the data center (see story). And if those new iPod Minis were just $50 cheaper, I'd have bought two by now.

However, the most underplayed news, and yet the most interesting thing to emerge from the show in my view, was Apple's new Xgrid application. This software is aimed at scientific applications like gene sequencing, missile-trajectory calculations, cryptography and other incredibly compute-intensive applications that previously could be crunched only by supercomputers.

Apple has already proved it can deliver supercomputer technology with a cluster installation at Virginia Tech (see story), so what does Xgrid bring to the table?

In typical Apple fashion, Xgrid provides an easy-to-implement solution for two common problems in the scientific community: the lack of cheap computing power, and wasted cycles on unused desktops.

Think of Xgrid as a customizable SETI at Home software package for serious computing folk. The Xgrid software allows you to tie together any number of desktop machines (Apple suggests using those with G4 and G5 processors) to work as a single machine to process a job. When one of the computing hosts is idle, it can join the 'hive' and process part of the currently running job. For large research institutions, which often have huge student labs that are mostly idle from midnight to 6 a.m., this is a tremendous computing resource that can now be harnessed on the Mac side.

The software also allows for anyone with a modicum of programming knowledge to create his own plug-in enabling the Xgrid software to process custom jobs. Scientist/programmers are quite common in the biological sciences field and would be very comfortable with a Unix application that they can easily customize such as Xgrid. In fact, see it for yourself at Apple's Advanced Computation Group Web site.

I never write about something without taking it for a real-world test drive, so I downloaded the Xgrid software last week -- though it took me a few tries. The download link didn't respond in the Safari Web browser, nor could I access it from the Finder, so I tried a few other tricks, which you may need to use in order to get the software.

In the Finder, do a connect to server (Apple-k keyboard shortcut) at ftp://ftp.apple.com/developer/. Then go into the directory called tool_chest. Inside, you'll see the Xgrid .dmg file. For some reason, I couldn't FTP directly and had to use this method to get the software. So if you have problems with the download, try what I did. It's worth it to get your hands on this software.

Getting the software was the hardest part of setting up Xgrid. Once I had the .dmg mounted on my desktop, I installed the client on my 15-in. Titanium Powerbook and also on my server, a Power Mac G4 (dual 533-MHz) running Panther server. Both were set up as clients, and for simplicity's sake, I didn't require authentication. It's important to note that you can secure the Xgrid service so that only authorized Xgrid controllers can use the CPUs in their cluster. This prevents "rogue clusters" from appearing in your computing environment.

Xgrid -- A first look at Apple's new cluster software

Since both of my nodes were on the same subnet, I was able to use Apple's Rendezvous software to configure the cluster. If you intend to access machines on other subnets, you'll need to enter the IP address or DNS name of the cluster host the service should bind to on those nodes.

To test the software, I used the built-in jobs directing the cluster to process a Mandelbrot fractal -- first using only one node and then using both. The processing time for the job was, as expected, cut in half. You can see both nodes listed below in the X-Blast screenshot and the summary of available power in the "tachometer."

I can't stress enough how easy it was to set up this software and get Xgrid up and running. Total time invested was under an hour, including downloading the software and uninstalling it after my tests. This is a great application for any scientific computing environment wishing to take advantage of unused computing cycles or wanting to achieve some form of supercomputing on a budget that doesn't allow for InfiniBand cards and multiple racks of Xserves.

Xgrid -- A first look at Apple's new cluster software

Lastly, it occurred to me that this Xgrid could be used in 3-D graphics processing for applications that are compute-intensive such as MAYA and SHAKE (see postscript below). I can see this as a real boon to many postproduction shops that have unused cycles in their editing machines that could be used for rendering during off hours. I asked the Xgrid folks at Apple about this, and while they stressed that Xgrid was developed for the scientific community and is currently geared toward it in terms of usability and interface, they were upbeat about the prospect of working with developers of these graphics applications to create an interface for Xgrid. I suggest the folks at Alias Software give them a call!

Now if they could only find a way to link five 40GB iPods together, I could carry my entire music collection everywhere I go!

Postscript: I've received a tremendous amount of mail regarding the Xgrid technology since posting this story. However, it seems I wasn't clear on the subject of distributed rendering for graphics. I'm quite familiar with the batch and network rendering options in Maya and Shake. However, I was referring to the ability to conscript the CPU cycles of another machine without having to install the application or its rendering engine on the conscripted machine -- thus avoiding the need for another license.

In this scenario, an Xgrid client could have multiple plug-ins (such as FCP, Maya and Photoshop) to interface with a licensed application simply for rendering. The features available are dependent on the writing of the front end in the application itself. I hope this clears up any confusion.

Do you have an opinion on this article? Think I missed something? Send your questions, comments and curses to Y.Kossovsky@ieee.org.
Looking for more Macintosh news? Be sure to sign up for
Computerworld's biweekly Macintosh newsletter.

5 power user tips for Microsoft OneNote
Shop Tech Products at Amazon