Seeing voices

VoIP monitoring tools have evolved, making large-scale deployments over converged voice/data networks manageable

When Kevin Lopez began sending voice calls over his IP data network using a new IP telephony system two years ago, he had no way of monitoring voice traffic to maintain the quality of calls. "We didn't have any monitoring on the voice side, so we were totally reliant on our [data] network counterparts," he says.

Now, using Avaya Inc.'s Integrated Management software, he can configure a softphone -- an applet that lets a PC function as a telephone -- place a call between any two points on the network and hear the voice quality while tracking network metrics on screen. "It's almost like a speedometer," says Lopez, national manager of telecommunications at Grant Thornton Inc., a global accounting and auditing firm in Chicago with about 3,500 users on voice-over-IP systems. "[It] will show you in red, yellow and green where the call was in terms of lost packets, or any sort of error it encountered." The tool can also "roll back" the speedometer to view and diagnose network conditions at the precise moment when a user says he was having trouble.

VoIP still requires special tools and skills because voice traffic is far more sensitive than data to common problems such as dropped or delayed packets. But as tools for managing VoIP traffic have evolved, Lopez and other VoIP managers have become increasingly confident that they can manage even large-scale deployments of voice communications systems over converged voice/data networks.

Looking back at what he was capable of doing with the management tools that were available when he first deployed VoIP, Lopez says, "Life is so much better. We can see troubleshoot our own [problems] ... and see exactly what's happening."

Network managers say the place to start is with network components and management tools that support Ethernet quality-of-service (QoS) standards for prioritizing voice traffic, and with management software that can do real-time, end-to-end monitoring of voice traffic across a LAN or WAN. Administrators should also do upfront evaluations of their networks to ensure that they can handle VoIP and learn to watch for subtle problems that don't interfere with data applications but wreak havoc with voice quality.

Voice traffic isn't a bandwidth hog, but on a converged network it can fall victim to unexpected, short-lived congestion caused by, say, a user initiating a streaming video application. One key metric to track is propagation delay. A lag of only 50 milliseconds can create echo in a call, while delays greater than 250 milliseconds can cause one speaker to talk over the other.

Jitter, the variation in arrival times of voice packets that should all arrive at the same moment, is another problem. One way to reduce jitter is to store and hold early-arriving packets until their slower counterparts arrive. However, making such "jitter buffers" too large can itself increase delays in the processing of the voice packets, reducing voice quality.

Designers of VoIP networks need to do upfront assessments to be sure their networks can handle the demands of temperamental VoIP traffic. They should also pay close attention to network performance metrics that barely mattered in the more familiar world of data traffic. But to do that, administrators need specialized tools.

Basic Requirements

The first requirement for any network management tool is that it must support QoS. Sean McRae, vice president and CIO at Prudential Northwest Properties, a real estate firm in Portland, Ore., uses Network Supervisor monitoring software from 3Com Corp. in Marlboro, Mass. The software "allows us to keep an eye on our WAN traffic between all of our servers and routers" to ensure that voice packets get top priority, he says.

The second requirement is the ability to perform real-time traffic monitoring so administrators can troubleshoot short-lived problems that cripple voice quality. Another common and useful feature is the ability to see the condition of the network at the point in time when the user reported a problem.

Network managers also need a tool that can monitor all of the network components along the call path that might affect voice quality. Lopez, for example, wants to see alerts not only for his IP private branch exchanges, but also for switches or any other component that is failing or overloaded.

Sources of management tools include the vendors that make network components such as switches and routers, as well as the manufacturers of IP telephony systems. For example, Nortel Networks Ltd.'s Optivity Policy Services and Optivity Network Management System provide central management and control of QoS across a network.

Third-party management software vendors also focus on VoIP management. Concord Communications Inc.'s eHealth Voice Quality Monitor assesses network readiness and monitors ongoing voice quality. The tools aren't cheap; a basic eHealth system starts at $6,000, and a more comprehensive package that manages QoS and the end-to-end network infrastructure starts at about $150,000.

NetIQ Corp.'s Vivinet Manager Suite provides proactive network and system monitoring and automatic alerting. It provides integrated monitoring of IP telephony, unified messaging and video applications. The suite also helps network administrators determine if voice problems are being caused at the hardware, operating system or application level. Pricing varies by customer environment, but a system capable of managing a 100-phone Cisco CallManager deployment costs about $6,000.

Another NetIQ application, the $5,000 Vivinet Diagnostics, finds the source of voice quality problems, identifies the causes and prioritizes them, combining network discovery, synthetic transactions and monitoring of LANs, WANs and network devices.

Because VoIP management is still a relatively immature area, management tools supplied by IP telephony system vendors such as Nortel, Cisco Systems Inc. and Avaya work best with their own products, says Robert Rosenberg, president of Insight Research Corp. in Boonton, N.J. And some management features may not work with gear from other vendors.

That can be a problem because most customers use VoIP technology from a number of vendors, says Darrell Epps, a technical assistance manager at NextiraOne LLC, a systems integrator and managed service provider in Houston. However, he says, over the past 12 to 18 months several independent vendors of management software have made progress in producing software that can manage VoIP components from multiple vendors. They include NetIQ and Concord as well as Micromuse Inc., which offers a system called Netcool for VoIP.

Some equipment vendors are also teaming up with third-party vendors to provide management tools that aren't tied to a single vendor. For example, Nortel includes NetIQ's Vivinet Management as part of its Enterprise Network and Service Management capabilities. And the Visual UpTime Select performance management application from Visual Networks Operations Inc. ships with Cisco's routers and can also be integrated with system and network management tools such as Hewlett-Packard Co.'s OpenView, says Naresh Kannan, director of product strategy at Visual Networks.

And many VoIP tool vendors are working to combine VoIP functions with their data management products, or with those from other vendors, in order to provide a single view of both voice and data traffic on the converged network. For example, Nortel is shipping IP phones that support real-time monitoring of voice calls and send alerts of problems that could affect voice quality to the same applications that network managers use to monitor data traffic. This reduces the number of management consoles customers must buy and monitor -- and makes it easier to find and fix problems. McRae also uses his management software, Network Supervisor, to monitor both the data and voice side of his network.

Bruno Battochio, communications specialist at Falconbridge Ltd., a Toronto-based mining company, uses a combination of Nortel's Optivity management software and a traditional packet analyzer to monitor the performance of IP phones linked to a Nortel Business Communications Manager system in an underground mine. "We didn't mix and match equipment" from multiple vendors, Battochio says, adding that he wanted to avoid situations he has faced in the past where different vendors blamed one another for a problem instead of fixing it.

Looking forward, vendors hope to provide "self-healing" networks with tools that not only proactively find problems but also fix them automatically. By 2005, Nortel hopes to be shipping a version of Optivity that can automatically solve voice quality problems by resetting a network port or redefining the QoS settings implemented on a switch, says Clive Foreman, vice president of engineering for enterprise network and service management at Nortel in Brampton, Ontario.

John Montgomery wants management tools that can automatically reroute voice packets if a slowdown hits part of the network, something that he says is now "a highly manual process." Montgomery, vice president and chief technology officer at Embarcadero Systems Corp. in Alameda, Calif., is about one quarter of the way through a plan to move 1,000 users to Cisco's IP telephony system. Although each upgrade of CiscoWorks management software brings more features, he says proactive troubleshooting features have been slow to materialize.

Despite some limitations, most observers says today's VoIP management software is up to the task of managing large-scale VoIP deployments -- although it can cost anywhere from $25 to hundreds of dollars per user per year, depending on the number of users and the range of functions required, says Ronald F. Gruia, an analyst at Frost & Sullivan Ltd. in Toronto.

"The management tools we have today are designed for installations of tens of thousands, if not hundreds of thousands, of users" says McRae. But choosing the right tool, and knowing how to use it, requires an understanding of the finicky new world of VoIP.

Scheier is a Computerworld contributing writer in Boylston, Mass. He can be reached at rscheier@charter.net.

Related:

Copyright © 2004 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon