Home > Security

Computerworld - One night last week, I was rushing to get home at a decent time when the head of our software support team slipped me a note suggesting that I look into an issue with data loss on one of our production database servers. I was tempted to just leave it until the next day, but I decided to investigate right away to obtain the highest-quality data from the fresh crime scene.

I quickly discovered that there was little hope of preserving evidence. The data had been deleted six hours before my arrival. During that time, the support team had run a partial investigation, but like any support group, the staffers had focused primarily on diagnosing the cause of the loss and restoring service. While they succeeded in restoring service, their work had made evidence preservation impossible.

Pieces of the Puzzle

The problem was that all of the tables within the database had been dropped and then re-created in a 10-second period. This erased all of the data in the database, including financial data that must, by law, be kept for seven years. The very quick deletion pointed to a script-based attack rather than a manual one.

The support team had found a prime suspect—the install scripts for the software that uses the database. They drop and re-create tables in the same order in which they had been deleted. But who had run the script?

SECURITYFIRST-AIDKIT After our mysterious database crash, we issued a “first-aid kit” to our software support team as a way to protect evidence during future incidents. We stitched the following components together into a single script that the staff can run: Fport, free from Mission Viejo, Calif.-based Foundstone Inc., identifies which applications are listening on which ports. Fport is a great way to uncover Trojan horses installed on your system. PSLoggedOn, an applet available free from Austin, Texas-based SysInternals LLC, provides a comprehensive listing of who’s logged on to a Windows NT system at a given time. Microsoft includes the Netstat utility with its Windows and .Net operating systems. We use it to list all ports and systems engaged in communication with a Windows NT server.

Our preferred response to a possible system compromise is to pull the plug and image the disk. After a few weeks of careful analysis, we wipe the disks and reinstall. When something goes wrong on a production system, we must leave it running for as long as possible, until we have compelling evidence that something awful is happening.