Top 7 Log Management Pitfalls to Avoid

By January 24, 2011Log Management, Security

As a security practitioner, log analysis is probably one of the most important parts of my job. Just as doctors check the history of their patients before they check their current vitals such as temperature and blood pressure, security practitioners review and interpret log history along with what is currently being reported. Armed with this information, we’re better able to diagnose the issues we find. Also like going to the doctor, we can’t put off log management until there is a security incident—by then, it could be too late.

I’ll admit, log management isn’t the most enjoyable job to do. However, since it is an integral component of many compliance regulations such as Payment Card Industry–Data Security Standard (PCI-DSS) and Sarbanes-Oxley (SOX), it’s important that we remain vigilant. If we don’t, we risk security breaches and loss of reputation, as well as failed compliance, which results in fines and sanctions.

When implementing a log management program, here are the top 7 pitfalls to avoid:

1. Not Logging at All (or Limited Logging)

Hopefully, the procedure for installing new devices such as a server, router, firewall and etc. on your network includes turning on logging or using more than the minimum. This could cause some angst with others in IT (especially database administrators) but we have to measure the risks vs. performance where security is concerned.

2. Logs Aren’t Reviewed

Once we having logging turned on, I’m willing to bet that there are few people that actually look at the logs on a regular basis. Many people that I’ve worked with only look at logs when they’re attempting to diagnose a problem. The reason for this is because logs are really cumbersome (boring) to read in its native format. You have to weed through all the normal events and hope that anything abnormal magically pops out at you. The person reviewing the logs must understand what’s contained in the logs as well as be able to determine when some action needs to be taken.

3. Reviewing Only a Limited Scope

If my router loses connection, how do you find out what caused it? You might check the logs and review what happened 10 minutes prior to the outage. If you don’t find anything, you might think it was a fluke. What if this port loses connection at the same time every week? How would you know? How long would it take for you to figure this out? This would take intimate knowledge of your environment and logs as well as a system to record such outages.

4. Logs Are Not Centralized

With all the different types of devices capable of creating logs, it is daunting task to log into each of devices and review the logs. Not to mention trying to mentally correlate all these devices in order to draw conclusions as data passed through various devices. With all the differences in formats between firewalls, intrusion prevention systems, routers, servers and other log devices it is very important to have logs in a format that can be easily searched and correlated in order to find that proverbial needle in a haystack. Centralized logging will help to ensure log format standardization and make it easier to correlated events among various devices.

5. Keeping Logs for Only a Short Period

Unfortunately, many security practitioners don’t figure this out until an incident happens and they discover that the problem had been going on for months. If the logs rotate after 30 days, the organization is handicapped when trying to piece together the severity of the breach. To make matters worse, regulations such as SOX and HIPAA are now requiring companies to keep logs anywhere from 6 months to 7 years.

6. Focusing on Compliance Instead of Security

Many companies decide to implement log management because of the compliance regulations they must comply with in their industry. However, simply implementing a product does not ensure that any particular compliance requirement as been met adequately. Most regulations also require some level of real-time monitoring and response. These regulations are put into place to identify and prevent breaches to sensitive data. This can’t be done by simply dropping a box on the network. Without real-time threat management companies are still leaving their data exposed.

7. Not Using an Automated Tool

In order to have an effective log management program, it is important to have an automated process that collects, normalizes, alerts and reports on logs. In these economic times, compliance mandates are growing but we don’t have the extra man power or time to implement another tedious, manual process. There are numerous tools on the market with very different focuses. There are log management tools that only act as a centralized repository without the ability to normalize the logs to make them more searchable and manageable. There are also products that may collect and normalize but don’t have the ability to alert and report on security issues or abnormalities. Generally, tools classified as Security Information and Event Management (SIEM) will fit the bill for the full automation of log collection, alerting and reporting.

To conclude, log management may be a necessary evil for security practitioners but extremely important in helping us to be successful in our jobs. When implemented and used correctly, we can not only satisfy our compliance mandates but also ensure that our corporate data is kept safe.

(Note: photo by OliBlac)