Snort Enhancements Suppress IDS Headaches

Computerworld - False positives and data management are two of the most frustrating aspects of using an intrusion-detection system, but a new version of Snort, the freely available IDS that we use, has greatly improved the situation.
We have more than 15 IDS sensors, and a full-time person is needed just to manage the IDS infrastructure, including the constant deluge of alert data that the sensors generate. Although we configured the sensors to send their alerts to our Security Information Management (SIM) software, a product from Edison, N.J.-based netForensics Inc. that does event correlation and analysis, the sensors are still the first line of defense.
Data management starts with the sensors. SIM applies additional correlation methodologies to the events to generate more precise reporting and additional alerting, but the individual network sensors generate the actual alerts. The sensors do this by analyzing the network traffic and matching the pattern of activity to the hundreds of rules we set up in our Snort rule base that trigger alerts on suspicious activity.
The problem is that we must continually tune the sensors to our constantly changing environment. And one of the most difficult aspects of managing the IDS infrastructure is tuning the alerts to reduce false positives. In our network, for example, we have a lot of Web-based activity. Even within normal network traffic flows, IDS sensors generate alerts on activity that looks suspicious but is actually benign. When presented with these false positives, we have a choice: either configure the sensors to stop sending alerts about the event, which means deleting or "commenting out" the underlying rule from the rules database; or take up disk space and bandwidth by storing every instance of the event.
The latest release of Snort includes thresholding and suppression features. I suspect that these improvements will spur new sensor deployments while making existing installations easier to manage.
Thresholding, a common feature in commercial IDS products, helps manage false positives. We forcibly prevent our IDS sensors from processing many types of events because of the abundance of false positives these rules produce.
For example, we see many alerts whenever a person or program attempts to access the robots.txt file on each of our Web servers. This file provides a way for our Web servers to tell search engines which areas of our Web sites shouldn't be accessed. Our Web servers contain confidential or sensitive information that we don't want indexed on Yahoo or Google. Many applications and Web browsers attempt to access robots.txt as part of normal operations. But