Cloud control systems tame the ether

Signifying a formless haze of computing power and storage that is somewhere "out there," computerdom's current buzzword is as difficult to get one's arms around as a real cloud. A seemingly limitless pool of processors and memory and disk space, and you just scoop out what you need. Sounds great, doesn't it?

Of course, it's not that simple. You don't just grab a handful of virtual computers and toss them after your data. They have to be configured. If you need an n-tier system, you'll have to assemble the virtual networking components to tie them all together. In short, managing a herd of virtual systems in the cloud is no less difficult than managing a herd of physical computers.

[ Is your head in the clouds? Don't be fooled. See InfoWorld's Busting the nine myths of cloud computing and What cloud computing really means. ]

The four products in this roundup are designed to simplify the life of a cloud-based virtual-machine wrangler. To varying degrees, and in varying environments, each provides tools for creating, deploying, and managing collections of cloud-machines.

The products are the AppLogic platform from 3tera, the Elastic Computing Platform (ECP) from Enomaly, the Infrastructure and Middleware on Demand (IMOD) system from Kaavo, and RightScale, from the company of the same name. The 3tera and Enomaly offerings are tools for building your own private or hosted clouds. The Kaavo system works with third-party cloud services such as Amazon EC2. And RightScale works with both private clouds and third-party cloud services.

3tera AppLogic

The 3tera AppLogic system is its own cloud. That is, you do not run it on a third-party cloud provider such as Amazon EC2; you run it on hardware hosted by 3tera, or you provide the hardware and 3tera builds the cloud. AppLogic is referred to as a "meta-operating system." At its heart, it is Red Hat Linux running a specially configured Xen hypervisor. More precisely, it is a cluster of interconnected systems running Red Hat Linux, Xen, and the AppLogic orchestration system.

AppLogic can run on a remarkably spare piece of hardware. Minimum requirements are a 1GHz processor, 512MB of RAM, and 80GB of disk storage. Of course, this is the barest of beginnings; a single backbone of an AppLogic system can consist of up to 128 machines, and backbones can be combined to form overall networks of thousands of machines. Assuming you don't have the space in your basement for your own AppLogic system, 3tera already has datacenters in North America, South America, Europe, Japan, and Singapore. (Quantities and locations are changing; you should check 3tera's Web site for the latest details.)

Though AppLogic's core is Linux, it can host virtual machines running Linux, Solaris, and Windows Server 2003. But the fundamental building block of a multitier AppLogic system is not simply a guest operating system; it is the duo of virtual machine (running the guest operating system) and application -- such as Apache or MySQL. This combination is called an "appliance," and it is a unique feature of AppLogic that, in an appliance, the application takes precedence over the containing OS. You create a cloud application by wiring together applications, and the fact that each application is running in its own OS is merely incidental.

This is a powerful paradigm, because it insulates you from the details of a particular OS; the OS is merely a container, a runtime environment for the application. Configuring and tuning the application are what counts, and this focus on application construction becomes apparent when you step inside AppLogic's management console.

[ Cloud computing offerings differ in depth, breadth, style, and fine print. See Cloud versus cloud: A guided tour of Amazon, Google, AppNexus, and GoGrid. ]

Log onto AppLogic's management console, and you are presented with the tool's Dashboard, where in-flight instances are displayed, accompanied by each instance's basic usage statistics: CPU, disk, and memory. Click the Applications tab, and you're shown a list of your account's applications; the list includes the application's state (stopped or running), a description, and associated CPU, memory, and bandwidth resources. Finally, the Support tab is a jumping-off point for 3tera's documentation, release notes, and support forum.

All the action is on the Applications tab. Select an application and the AppLogic editor opens. The application is represented as a network of nodes on a canvas. Each node is an appliance, populated with input and output connection points called "terminals." If you want, say, a Tomcat appliance to send database queries to a MySQL appliance, you drag and drop each onto the canvas, then drag a connection line from the output terminal of the former to the input terminal of the latter. It's a lot like wiring circuit components. An "appliance palette" waits to the side, from which you can drag new appliances onto the canvas and wire into your application. Right-click on an appliance, and you can configure its property values and attributes.

To actually configure or tweak the internals of an appliance, you log onto it. (Remember, an appliance is an OS running an application.) A secure shell window opens, from which you can execute Linux commands to your heart's desire.

The glue that cements appliances into applications is ADL, the Application Description Language. ADL is very much like XML, but less repetitive and devoid of brackets. It is used to describe all the components (appliances) that comprise an application, as well the structure of the appliances themselves. A complete description of the language is available on 3tera's AppLogic wiki.

3tera's version of cloud services is a "virtual private datacenter" (VPDC), executing on hardware hosted by 3tera itself. Pricing for a VPDC depends on a mixture of factors -- CPU, RAM, and storage requirements. Or, if you already have hardware in place and want to construct your own AppLogic installation, you can purchase an Enterprise AppLogic License. In either case, you should contact 3tera for details.

Enomaly ECP

Enomaly's Elastic Computing Platform (ECP) is not a tool for deploying to existing clouds such as Amazon Web Services. Like AppLogic, Enomaly ECP is a tool for building your own clouds. It is erected on a set of open source virtualization applications and APIs. You can construct your own cluster of systems, install ECP, and use its UI to manage the configuration, storage, and deployment of virtual machines. At its core is Enomalism, a virtual-machine management system written in Python that uses MySQL as back-end storage.

[ Ready to take a step into AWS but not sure where to begin? Read InfoWorld's "Inside Amazon Web Services" and "Hooking your apps into Amazon Web Services."]

The ECP user interface is a console that operates the mechanics of the tool's underlying system, which is, in turn, undergirded by the open source libvirt virtualization API. libvirt is a C toolkit that allows applications to communicate with the Linux kernel's virtualization capabilities, and thereby control hypervisors running on the system. The hypervisor is the virtualization software that allows a computer to host one or more OSes -- each in its own virtual environment. Currently, libvirt supports Xen, QEMU, KVM, VirtualBox, and others. (For more information, see libvirt.org.)

Currently, there are three versions of ECP: the free, community edition (which I tested); the Enterprise edition; and the new Cloud Service Provider edition, which adds usage accounting and billing integration to the user interface. Check the Enomaly Web site for details of the editions' differences.

ECP's management console is arranged along the same lines as the other consoles in the roundup. It is browser-based, with tabs for each of the major functions. The console opens to the obligatory Dashboard, which is really a transaction monitoring page. All operations performed in the console are transactions. They can be issued asynchronously, and some may take minutes to complete, so the Dashboard lets you monitor their progress.

The Virtual Infrastructure tab leads to three control areas. First, the Infrastructure control provides a navigation pane for all the clusters in your cloud. Select a cluster, and you can view all its assigned virtual machines. Buttons across the top let you start, stop, pause, or delete a virtual machine within the cluster.

Next, the Virtual Network Manager control panel assists in creating and managing virtual networks within your clusters. (This control is not in the community version; it was in a testbed system that Enomaly gave me temporary access to.)

Finally, the Elastic Valet is a control panel for provisioning machines. You select a target cluster for the machine, and specify launch parameters. The Elastic Valet will determine which physical machine in the cluster is the best destination to "park" the virtual machine -- hence the tool's name.

[ Application development has taken to the skies. See the InfoWorld Test Center reviews, "Windows Azure Services Platform gives wings to .NET" and "Google's high-flying cloud for Python code." ]

Another important component of the ECP control panel is the Repository, which is a storage management system for OS images and plug-ins for the Enomaly system itself. (The latter is typically supplied as Python eggs.) The Repository recognizes both local and "remote" machine images. Remote images are available from Enomaly's central servers; you can select a remote image and have it downloaded to your local system. At the time of this writing, there were about 10 images, including Red Hat, Ubuntu, CentOS, and others.

Hard-code developers will enjoy the Enomalism API selection from the Admin tab. This provides a list of the REST-style APIs that Enomalism supports. Select an API, and you're provided with a structured definition of the parameters and data types for that API.

ECP is simply not there yet. Or, if it is, it hides the fact behind its meager documentation. Though I was able to install and run the community edition on Ubuntu, whenever I got something to work, I could never be certain whether I had done the right thing or had simply stumbled into a solution. More importantly, ECP appears to be lacking in tools that assist in the configuration of machine instances. That is, it has nothing analogous to RightScale's RightScripts or Kaavo's system definition file. For now, it appears to be good for managing virtual machines in a cluster of networked hardware, but provides little support for applications on those virtual machines. Hopefully, this capability will appear in a future release.

Kaavo IMOD

Kaavo's Infrastructure and Middleware on Demand (IMOD) seeks not only to abstract the individual servers running on cloud-based system, but -- as much as possible -- to erase the boundaries between tiers. That is, with IMOD, you can work with a multitier application as though it were a monolithic application, without losing the ability to fine-tune constituent parts.

All the information needed to deploy an application is gathered into a single file, called the system definition file. Crack the file open, and you'll find what looks like a description of a state machine. The IMOD engine executes this file, booting the different tiers of an application in proper order. The file can also specify corrective action to be taken if a tier somehow fails to boot. Each tier specified in the system definition file consists of one or more servers. When you configure a particular tier, you specify the number of servers on that tier, and IMOD lets you manage them as though they were a single system.

The system definition file contains two major sections. One defines the static artifacts of an application (tiers, servers). The other is a flow control section that specifies the actions that the IMOD engine is to perform when the application is being launched. These actions are defined in the form of Velocity templates -- Velocity being a scripting technology that allows you to embed Java classes in a script file, so you can create scripts that are not only descriptive, but executable as well. It is this "descriptive executability" that permits IMOD to roll everything it needs to know about a system into a single file.

[ Simple, hosted, database-driven Web apps: See the InfoWorld Test Center review of Caspio Bridge, "A Web-based app builder with a Microsoft twist," and "Application builders in the sky." ]

IMOD's user interface is a three-tabbed control panel. Foremost on the list is the N-Tier tab, where you can view available templates and systems (the latter being the configured and executable form of the former). Select a system, and you can view it either in graphical, "run-time" form, or as a text-only XML file (the system definition file mentioned above). You can also open a tree view of the system, which shows the state of each of its Amazon Machine Images (running/terminated), its instance identifier, firewall rules, and more. Click the "start system button," and the selected system is deployed and launched in accordance with the system definition file.

Kaavo provides a number of prebuilt templates that you can use to jump-start your own applications. These include a variety of JIRA configurations (JIRA is a popular issue-tracking system), a MySQL cluster system, a template for the PHP-based collaborative project management system phpCollab, and more.