Zen and the Art of Intrusion Detection

When you can't watch everything, tuning an intrusion system becomes more of an art than a science

I configured our intrusion-detection system (IDS) network sensors this week. For the software, I decided to go with Atlanta-based Internet Security Systems Inc.'s RealSecure. I'm familiar with the product, and with the limited resources I have in both my department and within our network operations center (NOC), RealSecure seemed to be the most appropriate choice at this time. The challenge lies in setting up the IDS so a security department with a staff of two can handle it.


This Week's Glossary

Promiscuous mode: A mode supported by some network interface cards in which the card can collect all packets that it sees on the network.

Phf hack: An old Apache 1.0 vulnerability that allows an attacker to execute arbitrary commands on a Web server.

Windows Red Button attack: A Windows-based attack that takes advantage of the fact that the "anonymous" user name is a member of the group "everyone" by default. Unsophisticated administrators may create shared resources that are available by default to this group.


Lance Spitzner's white papers are a great resource in securing your infrastructure. Although his documents on securing Solaris and Windows NT operating systems are mainly geared toward firewalls, they make a great reference material.

Everything you ever wanted to know about SPAN ports can be gleaned from this most awesome document at Cisco's Web site.

Web site for Internet Security Systems, maker of RealSecure.

A couple of the nice features of RealSecure are its point-and-click Windows-like operation and the X-express auto-update feature, which I'll explain. As new vulnerabilities are discovered, attack signatures need to be incorporated into the IDS infrastructure.

In an ideal world, I would have a team of skilled security engineers writing or obtaining the signatures and reconfiguring each network sensor with the new attack signatures. But this is the real world. RealSecure's update feature automatically downloads and installs signatures to the master console as they're released. Then it's just a matter of pushing policies containing the new signatures from the master console to each of the IDS engines.

Simplicity Is Key

It's a very simple operation, and in my situation, simplicity is key. My plate is full: I am dealing with vulnerability assessments, SecurID token administration, antivirus efforts, a virtual private network, a firewall, Web trust audits, awareness training, abuse issues, policy development, physical security and more. I can't afford a highly managed IDS - not with only one other person and myself on staff.

Our internal network - like most networks these days - is switched Ethernet, which is a problem with the IDS. Back in the old days, before switched Ethernet hubs were popular, you could simply plug your IDS sensor into a spare port on a hub. After configuring the network interface card in promiscuous mode, you could "snoop" or "dump" all the packets destined to or from any other machines connected to the hub.

In a switched environment, however, once the switch learns the media access control (MAC) layer address of the interface card on a port, it forwards traffic for that MAC address directly to that corresponding port. The MAC address is the one that's burned into each interface card at the factory. An IP address can change, but the MAC address will always be the same for that card.

Supposedly, every card in the world has a different MAC address. Since network traffic is directed to a specific port, other ports on the switch don't see the traffic. Switch vendors have realized that there are legitimate reasons why one would need to see all traffic copied to one port.

For example, packet analyzers need to see packets for troubleshooting purposes. San Jose-based Cisco Systems Inc. has a feature called Switch Port Analyzer (SPAN), which allows the network administrator to configure a switch port to analyze other ports on a switch. It's a fairly new feature, and most network engineers have never had to configure a SPAN port. So when you start talking to engineers to get their assistance in configuring IDS sensors, you'll have to provide a little education.

I've decided to use Sun Microsystems Inc.'s Netra T1 systems configured with 1GB of RAM and an 18GB hard drive as my IDS engine. I installed a standard Solaris 2.7 operating system, and then tweaked it to make it secure. Included in my tweaking: disabling Telnet and file transfer protocol (FTP); installing Secure Shell (SSH) for encrypted sessions; and installing a SecurID token agent from Bedford, Mass.-based RSA Security Inc. for digital token authentication.

I then disabled any service or application that wasn't needed for the IDS application. I also did some kernel modifications to harden the TCP stack. After I got the software installed and everything configured, I installed the Tripwire file integrity checker from Tripwire Systems Inc. in Portland, Ore. I do this when I know I have a pristine system, untouched by the Internet. I believe that it would be very difficult to hack my IDS engine.

The Netra systems are nice because they come with two built-in network interface cards. I like to configure one of the interface cards in what's called "stealth mode." All stealth mode really means is that the interface is invisible to devices on the Internet. To do it, I configured the interface card with an IP address of The interface can't really route traffic, but it can be put into promiscuous mode and collect traffic.

I give a real IP address to the secondary interface card. It's this interface that serves as my administrative port for configuration and communication to my centralized monitoring and management station that resides in the NOC. Most vendors have made the initial installation and configuration of IDS software easy. For RealSecure, it's as simple as obtaining the software and executing the pkgadd command.

The most difficult aspect of configuring the IDS is tuning the engine and effective incident response. RealSecure gives you the ability to choose from among hundreds of events and attacks to monitor. The more stuff the software has to inspect, the less performance you're going to get. In addition, managing "false positives" is very time-consuming. The less your IDS must look for, the fewer false positives you'll have to deal with.

The Secret Sauce

Here's my technique for IDS configuration and tuning. (Many readers will disagree, and if so, I would be happy to battle it out online in Computerworld.com's Security Manager's Journal forum.) I position my network engines to monitor certain functional areas of my infrastructure. One engine will monitor my Oracle servers, another will monitor the front-end Apache servers and so on. For example, I now know that my front-end Apache servers are all running Solaris 2.8 with the latest patch cluster and the latest versions of Apache, WU-FTP and SSH.

Why, then, would I need to monitor for the old phf hack, which only affected Apache 1.0 back in late 1997? And why would I need to watch for the Windows Red Button attack? I have no Windows NT servers on this particular network.

Some would say that it's important to watch for attempts and perform event correlation. Again, if I had a small army of analysts or could afford a third-party response capability to watch all of the events and perform the event correlation, then I would watch for everything. I tune my engines by choosing attack signatures that represent a real threat to my systems. I can't concern myself with door wigglers.

As I'm running out of space, I'll discuss incident response next time.


Copyright © 2001 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon