Five mistakes of log analysis

Anton Chuvakin, netForensics Inc.   Today’s Top Stories   or  Other Security Stories  

Why SaaS is Vital to Email and Web Security Bringing Order and Security to your Mobile Workforce: Corporate Mobility Policy and Device Management Achiving Compliance Through Good Governance Magic Quadrant for Application Delivery Controllers Should You Install Messaging Security Software on Your Exchange Server? Ninja: Total Email Security Learn-Fast Guide: Software as a Service is Growing Up SaaS Solutions for Remote Systems Management Learn-Fast Guide: Software as a Service is Growing Up Sign up to receive Security Resource Alerts

October 21, 2004 (Computerworld) -- As the IT market grows, organizations are deploying more security solutions to guard against the ever-widening threat landscape. All those devices are known to generate copious amounts of audit records and alerts, and many organizations are setting up repeatable log collection and analysis processes.
However, when planning and implementing log collection and analysis infrastructure, the organizations often discover that they aren't realizing the full promise of such a system. This happens due to some common log-analysis mistakes.
This article covers the typical mistakes organizations make when analyzing audit logs and other security-related records produced by security infrastructure components.
No. 1: Not looking at the logs
Let's start with an obvious but critical one. While collecting and storing logs is important, it's only a means to an end -- knowing what 's going on in your environment and responding to it. Thus, once technology is in place and logs are collected, there needs to be a process of ongoing monitoring and review that hooks into actions and possible escalation.
It's worthwhile to note that some organizations take a half-step in the right direction: They review logs only after a major incident. This gives them the reactive benefit of log analysis but fails to realize the proactive one -- knowing when bad stuff is about to happen.
Looking at logs proactively helps organizations better realize the value of their security infrastructures. For example, many complain that their network intrusion-detection systems (NIDS) don't give them their money's worth. A big reason for that is that such systems often produce false alarms, which leads to decreased reliability of their output and an inability to act on it. Comprehensive correlation of NIDS logs with other records such as firewalls logs and server audit trails as well as vulnerability and network service information about the target allow companies to "make NIDS perform" and gain new detection capabilities.

Some organizations also have to look at log files and audit tracks due to regulatory pressure.
No. 2: Storing logs for too short a time
This makes the security team think they have all the logs needed for monitoring and investigation (while saving money on storage hardware) and then leading to the horrible realization after the incident that all logs are gone due to its retention policy. The incident is often discovered a long time after the crime or abuse has been committed.
If cost is critical, the solution is to split the retention into two parts: short-term online storage and long-term off-line storage. For example, archiving old logs on tape allows for cost-effective off-line storage, while still enabling future analysis.
No. 3: Not normalizing logs
What do we

Continued...
1 | 2 | 3 | NEXT  

Print this Story

Send Us Feedback

E-mail this Story

Digg this Story

Slashdot this Story

"Yes, NASA has confirmed that some laptops taken to the International Space Station were infected with an online-gaming password stealing..." Read more... "Linux is more secure than most operating systems, but Not if you don't practice basic security measures..." Read more... Read more Security posts or See all Blogs

Cellular operators say they're ready for Gustav Psystar calls Apple a 'monopoly' in antitrust charges Doubt cast on Seinfeld as Windows TV ads near More top stories...

IT workers hit hardest by offshore outsourcing, survey finds Microsoft: No more Windows Live Mail crashes with IE8 Beta 2 Microsoft warns of IE8 lock-in with XP SP3

Telecommuting: 6 questions to ask before you say yes Telework can change office dynamics in ways you hadn't anticipated. Proceed cautiously. Wi-Fi tweaks for speed freaks Got a painfully slow connection or random dead spots? Our tips will help you get the most out of your wireless network. 5 ways to drive your best workers out the door Listen up, managers: Employees don't quit the job; they quit you. Ultraportable laptops: Their rise and possible fall Netbooks, ultraportables, mini-notebooks — whatever you call them, they've been grabbing headlines. Are they here for the long term or just a flash in the pan? Windows Vista A to Z Reviews, analyses, how-tos, visual tours, hot issues and predictions about Microsoft's new OS. More Continuing Coverage After 9/11: Homeland Security Apple's iPhone Ask a Premier 100 IT Leader Bill Gates steps down BlackBerry Legal Battle Data Security Breaches Electronic Voting Firefox H-1B Visas HP's Boardroom Scandal Handheld Devices Hurricane Katrina Aftermath Microsoft Legal Issues Microsoft's Vista Open Source RFID Technology SCO's Linux Fight Sarbanes-Oxley Act Spam Voice Over IP Wi-Fi Windows Meta File Vulnerability Windows XP Service Pack 3 IT Careers 2010 Four years from now, the IT field will be a vastly different place. Will you be ready? More Special Reports Premier 100 IT Leaders 2007 Best in Class 2006 Computerworld Horizon Awards 2006 Premier 100 IT Leaders 2006 Salary Survey 2007 Best Places to Work in IT 2007 IT Forecast 2007 Premier 100 IT Leaders 40 Years of Computerworld ASPs, Take Two Best Places to Work in IT 2003 Best Places to Work in IT 2004 Best Places to Work in IT 2005 Best Places to Work in IT 2006 Business Intelligence Home Runs Business Intelligence: Smarter BI Business Intelligence: The Future of BI CRM Goes Vertical CRM: Sober CRM Career Planning Guide 2005 Careers: IT Profession 2010 Computerworld Horizon Awards 2005 Data Management: Mining for Gems Data Management: Taming Data Chaos Development: Hard-Workin' Web Sites Development: New Tools New Choices Development: The New World of Application Development Development: The Web Services Tsunami Development: Web Services Hurdles Disaster Recovery: Preparing for the Worst E-commerce Grows Up E-mail/Groupware: Big Decisions Faces of Mobile IT Forecast 2006 Global Mobile Hardware: Multiple Cores, Multiple Challenges Hardware: On-Demand Un-Hyped Hardware: The Shape of Things to Come Horizon Awards: 10 Cool Cutting Edge Technologies IT Management: Guide to Managing Vendors IT Management: Navigating Global IT IT Management: Recovery Ahead IT Management: The Future of IT IT Management: The Resourceful Project Manager Innovative Technology Awards 2004 Management: Reinventing IT Microsoft in Transition Mobile/Wireless Leaders and Laggards Mobile/Wireless: The Untethered Worker Mobile/Wireless: Tiny Gadgets, Huge Costs Network Management: Taking Control Networking: Turbulence Ahead! Networking: VoIP Goes Mainstream Operating Systems: In the Slow Lane Operating Systems: Linux Goes Global Operating Systems: Should You Nix Unix? Outsourcing: Offshore Buyer's Guide Outsourcing: Outsourcing Dangers Premier 100 IT Leaders 2004 Best in Class Premier 100 IT Leaders 2005 Best in Class ROI: Do the Math! Salary Survey 2003 Salary Survey 2004 Salary Survey 2005 Security Action Plan Security: Compliance Headaches Security: Proactive Security Security: Risk and Reward Security: Tips From Security Pros Souped-up Security Storage: Battling Complexity Storage: Cheap & Secure Data Stores Storage: Stretching Your Storage Dollars Storage: The Lean Storage Machine Storage: The New Rules of Storage Storage: The Ultimate Backup Guide Supply Chain: Missing Links Supply Chain: RFID Reality Check The Business of Security Web 2.0 Security Wireless: Wireless At Work All Zones Application Performance Zone Business Continuity Zone The File Data Management Zone Security Management Zone ITIL Best Practices Zone The SAS Zone Business Intelligence and Analytics Zone Windows Protection Zone Identity & Security Management Zone See your link here

From Laggard to Leader: Transforming the Data Center From Laggard to Leader: Transforming the Data Center Register for this complimentary live webcast today! Go to the webcast  Computerworld Executive Bulletin: Building a Robust Antivirus Defense Download this Executive Bulletin (a $49.95 value) for free, compliments of MessageLabs. (Source: MessageLabs) Antivirus software alone isn't enough to prevent today's speedy, sophisticated virus attacks. Security managers should consider multitiered approaches that include behavior scanning, appliances that check e-mail for worms, and restricting user access to dangerous Web sites. Download this Executive Bulletin (a $49.95 value) for free, compliments of MessageLabs, to learn more. Download this executive briefing  Online Security Issues in Regulated Industries Download this research paper, free for a limited time, compliments of Webroot! (Source: Webroot Software) In June 2008, Computerworld invited IT and business leaders to participate in a survey on online security initiatives at their organizations. The goal of the survey was to better understand Web and e-mail security issues faced today within the regulated education, financial services, government and health care industries. The following report represents top-line results of that survey. Download this white paper  White Papers Read up on the latest ideas and technologies from companies that sell hardware, software and services. Archiving Compliance with Sunbelt Exchange Archiver The Impact of Messaging and Web Threats Advanced Load Balancing: 8 Things You Need to Handle Today's Network Traffic View more whitepapers

• Cellular operators say they're ready for Gustav
• IT workers hit hardest by offshore outsourcing, survey finds
• Psystar calls Apple a 'monopoly' in antitrust charges
• More top stories 

Security Management Zone Security management is the process of developing a comprehensive data protection plan. It takes into account all potential threats, the existing network environment, the future needs of the organization, and lays out a multi-tiered blueprint to integrate the security technology needed to combat these threats. CDW can help keep your network and data secure. Visit the CDW Security Management Zone now See All Zones

Fired up about IT? Join Sharkbait and share your true tales of IT. SharkBait is the place for you to sound off about everything IT – the good, the bad, and the rest of the weird stuff you deal with every day. New baits

Identity & Security Management Solve your compliance problems and mitigate risk. Automate, validate and enforce business governance with Novell Identity Management and Security solutions. Learn more in the Identity & Security Management Zone See All Zones

In Security Stripping away the trappings of applications, systems and networks, information is the core asset of most organizations. Our columnist describes how asserting the importance of information governance is crucial to making that asset tangible, addressable and protected. Click here to read the latest column by Jon Espenschied