Protocol analyzers: How to compare and use them

Protocol analyzers are the indispensable tools that your IT staff deploys on the network when it's not behaving properly. Sharp, experienced admins can examine the live network data or a saved packet-capture file and figure out why connections are intermittent, why users are complaining that the network is slow or they can't get to their file server, or why voice over IP isn't working in a branch office.

There's much more. While network gurus still run portable analyzers from their laptops to troubleshoot and debug issues as they crop up, enterprise-grade tools have emerged to continuously monitor the health and security of your network and the complex, latency-sensitive applications that are your business.

This Toolbox explains the role of protocol analyzers and the monitoring, analysis and visibility tools that help keep your networks and the applications they deliver running smoothly and securely. Read more implementation tips in Protocol analyzers: Dos and Don'ts (free Insider registration required).

Protocol Analysis in the Trenches

What we know as protocol analyzers were commonly referred to as (and are still sometimes called) packet sniffers, but Sniffer is actually a brand name for NetScout Systems products. (It's much like Band-Aid is used generically for adhesive bandages.) Further, protocol analyzers have evolved to a level that goes beyond just the packet-capture capabilities the term implies.

[Read about How to stress-test your network | Vulnerability management tools]

"The change in name is significant," says Mike Chapple, senior adviser to the executive vice president at the University of Notre Dame, because the tools become very protocol-aware. He draws an analogy between protocol analyzers and the evolution in firewalls. "You had packet-filtering firewalls, then stateful inspection and application proxy. Network analysis tools have had the same evolution, from just packets one at a time without any context, without understanding what came before and what came after. Now tools are context-aware."

Protocol analyzers capture data off a particular port or network segment using a spanning tool, reproduce it in something approaching readable form, and provide some level of analysis to highlight key information. They are primarily network troubleshooting and debugging tools to figure out what is causing performance issues, why protocol errors are popping up, why DHCP isn't working, why your virtual network isn't routing traffic properly, and related issues.

They are often used when a new service is introduced or an existing one is changed, which is when implementation and configuration errors are most likely to occur. In some cases, poorly written or incomplete documentation may be the culprit, so admins have to take a close look at the traffic to figure out exactly what is happening.

"We can't necessarily trust that the configuration you think is running is the actual configuration running, so actually grabbing the packets and looking at them tells you exactly what's happening," says Joel Snyder, senior partner at IT consultancy Opus One. "These are very, very important tools and are used constantly by the network manager and the people debugging and troubleshooting."

Portable protocol analyzers are primarily network tools but are also useful for troubleshooting and fine-tuning security products such as firewalls and intrusion-detection systems. For example, Chapple said he has regularly used the ubiquitous free analyzer Wireshark to troubleshoot firewall rules. With systems running an analyzer on either side of a firewall, one can see exactly which packets are passing through and determine what's causing access issues--authorized traffic that's not getting through, or unauthorized or potentially malicious traffic that is.

They're also very helpful in analyzing an attack, such as determining the type of denial-of-service (DoS) attack is assaulting a network so it can be effectively countered.

On the negative side, malicious users can deploy an analyzer to eavesdrop on enterprise traffic.

There are a number of free commercial analyzers available, such as Capsa Network Analyzer, Microsoft Network Monitor and the command-line tool tcpdump. There are also commercial analyzers from companies such as NetScout, but the free, open-source Wireshark has become something of a de facto standard for portable protocol analysis. Wireshark is noted for its filter language, user interface, support for more than 1,100 protocols, and detailed information on more than 90,000 protocol fields, according to its creator, Gary Combs, director of open-source projects at Riverbed Technology.

"Wireshark killed the market," says Snyder. "Companies that want to sell moved up the stack, doing more application-layer network knowledge. The old sniffer market disappeared."

Stepping Up: Enterprise-Scale Monitoring and Analysis

As valuable they are, portable protocol analyzers aren't designed to scale across the enterprise network, especially when one considers the challenges of such a network: enormous traffic volume; diverse and complex applications, from enterprise risk management to social media; speeds of 10Gbps and above; and performance and availability requirements for high-speed financial transactions, VoIP, video streaming, and so on.

"Even the slow networks of 10 years ago were too much to analyze manually," says Notre Dame's Chapple. "Our tools have become more sophisticated, just as we have become more sophisticated in way we manage things."

For example, Wireshark is integrated into Riverbed Technologies appliances that can be deployed at key points across the enterprise to monitor and analyze traffic on the entire network and, if needed, deep-dive with Wireshark from a console rather than having to be picked up and plugged in to analyze a problem.

"Wireshark distributed doesn't do a good job of storing packets," says Snyder. "It's good when you're looking at a transaction and need to look at 20 or 100 packets to figure out what's wrong with that transaction."

Similarly, NetScout has a Sniffer suitable for enterprise-scale analysis in its product line, and Colasoft sells an enterprise edition of Capsa.

The enterprise market includes a range of very powerful enterprise products and suites that focus heavily on application performance and issues on high-volume, high-speed networks, from companies such as Opnet Technologies, NetScout, HP, CA, Quest, Compuware, IBM, Oracle and Nimsoft.

[Also learn more about Network behavior analysis tools]

"It's typically not the network that's the problem," says Steve Shalita, vice president of marketing at NetScout. "It's the context of application flows that is really meaningful. With huge data streams, you need to automate and get a clearer view of transactions and applications rather than individual packets."

So, says Snyder, these tools will look at broad application statistics such as average HTTP traffic transaction time, DNS query and SQL Server response time, retransmission rates, and top talkers and listeners on the network.

Tools from other vendors, including Solera Networks, NetWitness, Niksun and Endace, are built to capture and analyze every packet that traverses the network, providing continuous monitoring and intelligence about the network, applications and users. They have a strong security play in addition to monitoring application and network health. The extensive and granular information these products also provide can be leveraged by other security tools, such as firewalls, intrusion-detection and intrusion protection systems, security information and event monitoring systems, and malware analysis. The vendors say this level of visibility is essential in dealing with complex security problems, such as advanced persistent threats, the Stuxnet worm, malicious insider activity, bots and sophisticated malware.

Bringing It All Together

In an old Saturday Night Live skit, a couple is arguing about whether a product called New Shimmer is a floor wax or a dessert topping. They're both right, the baritone announcer assures them. "New Shimmer is a floor wax and a dessert topping."

So, is the goal of a protocol analyzer to assess network performance, application performance or security? It's all three.

"We're seeing the emergence of the service-delivery manager," says Shalita. "That role is about orchestrating the traditional realm of network, applications and security into this notion of managing delivery of service to users."

So, when users complain that the network is slow, it may not be--and frequently is not--the network, but application issues or some type of security problem, such as a DoS attack or bot traffic. IT is increasingly regarded as a service provider to its user constituency, and the overall value proposition of analysis and monitoring tools is to minimize disruption in the delivery of applications and services-- and, as a result, disruption to the business.

This story, "Protocol analyzers: How to compare and use them" was originally published by CSO.

Copyright © 2011 IDG Communications, Inc.

Bing’s AI chatbot came to work for me. I had to fire it.
Shop Tech Products at Amazon