If a firewall is like having a security guard at your office door, checking the credentials of everyone coming and going, then an intrusion-detection system (IDS) is like having a network of sensors that tells you when someone has broken in, where they are and what they're doing.
Firewalls work only at the point of entry to the network, and they work only with packets as they pass in and out of the network. Once an attacker has breached the firewall, he can roam at will through the network. That's where intrusion detection is important.
There are a number of approaches that can be used for detecting intruders. Many experts advise using a combination of methods rather than relying on any single mechanism.
Host-Based Detection
Perhaps the most famous IDS is Tripwire, a program written in 1992 by Eugene Spafford and Gene Kim. Tripwire exemplifies the host-based agent approach to intrusion detection: Installed on a host, it checks to see what has changed on the system, verifying that key files haven't been modified.
The agent is initially installed against a pristine host installation and records important system file attributes, including hashes of the files. The agent software then periodically compares the current state of those files to the stored attributes and reports any suspicious changes.
Another host-based approach monitors all packets as they enter and exit the host, essentially taking a personal firewall approach. Receipt of a suspicious packet triggers an alarm. Other commercial host-based products include Cupertino, Calif.-based Symantec Corp.'s Intruder Alert and Issaquah, Wash.-based CyberSafe Corp.'s Centrax.
Network-based intrusion-detection systems scrutinize all packets on a network segment, flagging those that look suspicious. A network IDS searches for attack signatures - indicators that the packets represent an intrusion. Signatures might be based on actual packet contents and are checked by comparing bits to known patterns of attacks. For example, the system might look for patterns that match attempts to modify system files.
Other network attacks are protocol-based. Attackers often seek weaknesses in a network by probing for active but poorly administered Web, file or other servers. These port attack signatures are identified by watching for attempts to connect to network ports associated with services that are often vulnerable.
An attack with a header signature uses malformed or illogical TCP/IP packet headers. For example, an attacker might try to send a packet that simultaneously requests to close and open a TCP connection; such a packet might cause a denial-of-service event for some systems.
Commercial network-based systems include Cisco Systems Inc.'s Secure Intrusion Detection System (formerly known as Cisco NetRanger), Atlanta-based Internet Security System Inc.'s RealSecure and Symantec's NetProwler.
What You Know, What They Do
Detection systems can also be categorized as knowledge- or behavior-based. Most commercially available systems are knowledge-based, matching signatures of known attacks against changes in systems or streams of packets on a network. Such systems are reliable and generate few false positives, but they can detect intruders using only attacks they already know about. They're often helpless against new attacks, so they must be continually updated with new knowledge about new attacks.
A behavior-based IDS instead looks at actions, attempting to identify attacks by monitoring system or network activity and flagging any activity that doesn't seem to fit in. Such activities may trigger an alarm - often a false alarm. Though false positives are common with a behavior-based IDS, so is the ability to detect a previously unreported attack.
Another intrusion-detection tool is the "honeypot," a completely separate system designed to offer an attractive nuisance to attackers. One manager of a prominent Web site often uses a honeypot to handle all inbound requests. Any attacks against the honeypot are made to seem successful, giving administrators time to mobilize, log and track the attacker without ever exposing production systems.
Intrusion detection requires considerable planning. As with virus detection, host-based intrusion detection that monitors system and file changes must be installed on pristine systems. Otherwise, there's always the chance that the system has already been compromised prior to installation of the IDS.
It's even more important to have a clear procedure in place for dealing with intrusions. It's not always best to simply pull the plug once you know that an intrusion is under way.
Depending on what systems or networks have been compromised and what you want to happen to the attackers, it's often preferable to keep the attackers in the system and contact a law enforcement agency to try to catch them. Such a decision shouldn't be made in haste; a set of intrusion response policies and procedures should be prepared well in advance. You want to keep intruders out, but you also want to discover and locate them when they succeed.
Loshin is a freelance writer in Arlington, Mass.