InfoWorld review: Meeting the network security and compliance challenge

Previous 1 2 Page 2

Client agents can also be used to store events, in case the centralized log management tool is offline. One of the best features of the most sophisticated agents is in measuring network and/or local CPU utilization and throttling back the message send rate until the congestion clears up. Lastly, many agents have a "heartbeat" feature that will send warnings if the client has not transmitted messages in a certain time period, although this can be mimicked identically with "zero baseline" alerts as well server-side only. Not surprisingly, ArcSight, a longtime SIEM leader, has more client agents than any other competitor.

As covered above, the more parsed data a log management product has, the faster and more efficient the product can be when sifting through large amounts of data looking for a particular data interest. A big differentiator of products is how many parsers the product comes defined with. Some of the leaders, like ArcSight, are bundled with well over 100 data collectors defined. On the lower end, some products only have a few dozen parsers or will claim that their generic parsers are identical in efficiency. But in general, the closer the parsers mimic your environment, the better (but don't let this be the sole decision point). Some log management products allow administrators to create their own parsers, which could prove very useful in many environments.

One relevant additional note: Most products claim to have Windows event log collection agents. However, many of these agents were made prior to Microsoft's latest Windows versions and don't have a good understanding and parsing of the more granular logs and views in these later operating systems. Many of the parsers and agents understand the three conventional default logs -- Application, Security, and System -- but cannot allow the administrator to choose from among the 100-plus built-in views that Windows Vista, Windows 7, and Windows Server 2008 provide.

Splunk is one tool that understands the new Windows log formats. However, I didn't find a tool that works easily with the newer Windows' built-in, event forwarding technologies (even if the product was hosted on a Windows OS and easily could use the newer technologies). Windows' own event forwarding could be used in place of all the other agent and agentless methods. As with most product categories, log management hasn't kept up with the latest client changes.

Log storage. Storing tens of millions to billions of messages takes a lot of disk space. Most appliances come with terabytes of disk storage in RAID configurations. Both software and hardware products claim to do some sort of storage compression, but as with transmission compression claims, take the vendors' figures with a grain of salt. Their storage compression statistics are often based upon the smallest event log messages and the highest compression values, and don't reflect real-world results.

Still, it's important to find out from the vendor whether the product is software or appliance, what is the maximum disk space (or file size) the product supports, and in what configurations. What RAID arrays are supported? Different RAID configurations have different performance characteristics -- that is, some are faster at writing and some are faster at reading -- so flexibility is a plus. Does the vendor support digital signing of collected log data for attestation needs?

Most products have a maximum log size as well, having to do with limitations of the underlying host OS. If the product is an appliance, can the data be stored to external drive arrays? How much data can actively be indexed and easily retrievable? Every product allows data to be exported or archived. Exported data typically is kept offline and must be imported en masse to be searchable. A few solutions handle this more flexibly. For example, LogRhythm allows administrators to define a filter to import only the needed data instead of everything.

A few products have what is known as "storage groups," which are individually defined logical partitions devoted to a particular task, such as PCI compliance, or a particular grouping of devices -- for example, Cisco wireless routers. In addition to organizing a certain class of data for reporting purposes, storage groups can be used to make sure that a particular application has enough disk space to serve a particular policy requirement -- for instance, save data for two years. ArcSight is especially strong in this area, with sizing parameters and CPU prioritization available.

Lastly, you'll want to determine whether event log data is stored or archived in the vendor's proprietary format, in its raw (unfiltered and unstructured) form, or in both? Most products store active data in a proprietary format, but archived or exported data remains in a raw format. This means that re-imported data will have to be parsed and indexed again to be useful, but it's also easier to prove chain-of-custody concerns if that raw data (assuming it is also digitally signed) is later needed for legal reasons.

Real-time viewing. Most products allow real-time viewing of incoming data and show some top trends -- often called tailing. If you have a system of any moderate size, with hundreds to thousands of messages coming in every second, real-time viewing of all data quickly loses its allure. All products allow real-time data to be filtered to show only relevant events for a particular interest. Often these filters can be saved to search historical data and produce related reports.

The best real-time viewers allow users to click on specific data fields to pivot to new views. For instance, maybe you're viewing incoming data about a particular workstation and you see a suspicious TCP port. In some products, clicking on the port value could switch the current real-time view to show all workstations using the same port. Other products can only do this on historical data or require that you switch views into an "investigator" mode. All of the products tested provide pretty flexible viewing, though LogLogic and LogRhythm were strongest.

Searching stored data. Searching stored data for interesting patterns and events is an important part of log management, and an area where vendors strive to differentiate their products. Vendors will often tout how quickly their filtered searches work across very large amounts of data (although none of these claims were tested in this review). Most offer searches based upon keywords, English phrases, and Boolean logic. Some vendors force the user to type in all search expressions, while others also provide a graphical, pick-and-choose, "build a query" interface. Building a query click by click is helpful in teaching new administrators, although experienced admins almost always prefer the quickness and flexibility of a typed query.

If your organization needs to search a lot of raw, unstructured event logs, ask the vendor if they support search filters across non-normalized data. And if they do, how exactly can you search it, and how do searches of unstructured data differ from searches of structured data? Many vendors only allow keyword searches of raw data, whereas others allow Boolean logic.

Can searches be performed across peers? Among the products reviewed, only ArcSight, LogLogic, LogRhythm, and Splunk can execute searches across multiple nodes. All the products allow search filters to be saved. But the better products allow them to easily be turned into reports and saved for later use. Some products allow search filters to be sent to others and shared, which is particularly helpful in very large environments with many log reviewers.

It's also good to have plenty of built-in, predefined search filters. Some products come with none or just a small sampling. The best products come with dozens of predefined, interesting queries, typically tied to one or more compliance objectives. The most common are for failed logons. A few products, including LogLogic, include "near context" queries that will show 10 or so events before and after a particular message you are interested in.

Alerting. Alerting is an important feature of log management and even more essential for SIEM. The vendor should support several different methods of alerting. All the products reviewed have email alerting, and most allow SNMP forwarding. Surprisingly, only a few have SMS alerting or allow analog modem dialing for pagers that lack an Internet interface. Some, including NitroSecurity, interface with common help desk software (usually Remedy) or have their own "help desk" function to help with responses. Most products allow unlimited alerting, but some, notably ArcSight Logger, only allow a limited number of active alerts -- five in the case of Logger. ArcSight's SIEM product has no such limitation.

Alerting comes in several forms. At the very least, alerting allows a notification to be sent if a particular log event is detected, and all products allow alerts to be based upon a certain number of events in a particular time period. One of my favorite alert types is the baseline alerting, in which the product itself determines the "normal" event patterns for the environment, while the admin determines the percentage of deviation to alert on. NitroSecurity supports baselining on every message type, whereas LogLogic's baselining is limited to all messages coming from a particular device or set of devices.

Whatever log management product you choose, make sure it has the ability to throttle alert messaging. Nothing is worse than getting 100 alerts from a single event in the middle of the night.

Reports. All products come with built-in reports and allow reports to be customized or created. The best products come with hundreds of built-in reports, either free or for additional charge, relating to particular security or compliance needs: NERC, PCI, SOX, FISMA, and so on. Reports can usually be saved to a variety of formats -- CSV, HTML, XLS, TXT, and sometimes PDF -- run ad hoc, scheduled, and published to predefined file shares. The more built-in reports you have to work with, the better.

Be sure to test the reporting differences regarding structured versus unstructured data. Most vendors cannot easily handle unstructured data in reports or cannot provide the same summaries and counts as are normally available for structured data. Some vendors can incorporate unstructured data into reports only by including the complete raw message detail or very minimal summary counts.

In addition to middle and upper management reports regarding particular compliance initiatives, look for detailed reports that support technical troubleshooting. The products with the best reporting functionality, including ArcSight, LogRhythm, and NitroSecurity, meet both of these needs. Some vendors are operating on workflow processes where compliance reports can be sent up the chain of command and signed off by the necessary responsible parties. My advice is to find out what reports come built-in, what reports are available at additional cost, and to review all of them to see if they fit your compliance needs.

All seven products reviewed contained hundreds of features and proved immensely configurable, and every one represents a solid, well-thought-out solution to log management. I found myself really liking each product reviewed, only to be further impressed with the next product I tested. Read the accompanying product reviews, which highlight the significant differences, to find out which product most closely fits your environment. Then give it a detailed test-drive to measure suitability and performance.

If you aren't using a comprehensive, enterprise-wide log management solution already, you have a number of excellent products to choose from. The best solutions give you only the alerts you require, filter out the noise, and provide useful dashboards and reports that you can tailor to your specific needs. The better you become at log management, the better equipped you'll be to serve your company's information technology needs, whether those relate to security, compliance, operations management, or virtually any other area of IT.

• Connectors provide lots of flexibility and options
• Fast queries
• Plentiful reporting options

• Limited to five active alerts
• Large client agents

• An easy-to-use GUI
• A good value for small and midsize businesses
• A good number of predefined rules and reports

• Cannot perform holistic, keyword searches across all events
• Lacks enterprise features such as event compression, network bandwidth throttling, a command-line interface, and storage groups
• Performance not in the same class as enterprise-focused products

• Clean and simple interface
• Ability to identify incoming device streams
• Adaptive baseline alerts

• Lacks context-sensitive graphics in many areas
• Not as feature rich as the competition
• Limited number of alert notification methods

• Numerous data views and easy pivot tables
• Able to view and filter real-time data
• Strong Active Directory integration

• Initial install could be improved
• Cannot capture SNMP traps (since corrected by release 5.1)

• Very flexible console views and graph
• Feature-rich, data-driven
• Adaptive baseline alerting

• Auto-discovery is weak
• No SMS alerting
• GUI is a bit busy and complex

• Strong reporting on unstructured data
• Granular Windows log selection
• Ability to distribute functionality
• Granular user roles

• Some custom configuration options require XML coding
• Limited number of reports and searches in default Windows application
• Some features, like client certificate mapping, configured outside management console

• Combines log management and SIEM in one box
• Dynamic traffic maps
• Strong technical support

• Slow boot time
• Weak embedded help system
• Some minor technical issues

 

This article, "InfoWorld review: Meeting the network security and compliance challenge," was originally published at InfoWorld.com. Follow the latest developments in information management and security at InfoWorld.com.

Read more about data explosion in InfoWorld's Data Explosion Channel.

This story, "InfoWorld review: Meeting the network security and compliance challenge" was originally published by InfoWorld.