Security fail: When trusted IT people go bad

Computerworld - It's a CIO's worst nightmare: You get a call from the Business Software Alliance (BSA), saying that some of the Microsoft software your company uses might be pirated.

You investigate and find that not only is your software illegal, it was sold to you by a company secretly owned and operated by none other than your own IT systems administrator, a trusted employee for seven years. When you start digging into the admin's activities, you find a for-pay porn Web site he's been running on one of your corporate servers. Then you find that he's downloaded 400 customer credit card numbers from your e-commerce server.

And here's the worst part: He's the only one with the administrative passwords.

Think it can't happen? It did, according to a security consultant who was called in to help the victim, a $250 million retailer in Pennsylvania. You never heard about it because the company kept it quiet.

Despite the occasional headlines about IT folks gone rogue (remember Terry Childs, the network administrator who held the city of San Francisco's network hostage?), most companies sweep such situations under the rug as quickly and as quietly as possible.

An annual survey by CSO magazine, the U.S. Secret Service and CERT (a program of the Software Engineering Institute at Carnegie Mellon University) routinely finds that three quarters of companies that are victimized by insiders handle the matter internally, says Dawn Cappelli, technical manager of CERT's threat and incident management team. "So we know that [what's made public] is only the tip of the iceberg," she says.

By keeping things quiet, however, victimized companies deny others the opportunity to learn from their experiences. CERT has tried to fill that void. It has studied insider threats since 2001, collecting information on more than 400 cases. In its most recent report, 2009's "Common Sense Guide to Prevention and Detection of Insider Threats" (download PDF), which analyzes more than 250 cases, CERT identifies some of the most common mistakes companies make: inadequate vetting during the hiring process, inadequate oversight and monitoring of access privileges and overlooking of red flags in behavior.

But threats from privilege-laden IT employees are especially hard to detect. For one thing, staffers' nefarious activities can look the same as their regular duties. IT employees routinely "edit and write scripts, edit code and write programs, so it doesn't look like anomalous activity," Cappelli says. And they know where your security is weakest and how to cover their tracks. You can't rely on technology, or any single precaution to protect yourself from rogue IT people. You have to look at the big picture.

"It requires not only looking at what they are doing online but also what's happening in the workplace," says Cappelli. "People really need to understand the patterns here, the story behind the numbers."

Computerworld went looking for some of those stories behind the numbers, incidents that have not been widely reported. Though the victimized companies wouldn't talk, the security consultants who helped clean up the messes would. Although each story has unique circumstances, together they show some of the typical patterns that CERT emphasizes. Employer, beware.

Pirating software -- and worse

The Pennsylvania retailer's tale of woe began in early 2008, when the BSA notified it that Microsoft had uncovered licensing discrepancies, according to John Linkous. Today, Linkous is chief security and compliance officer at eIQ Networks, a security consultancy. His experience with the incident involving the retailer is from his previous job, when he was vice president of operations at Sabera, a now-defunct security consultancy.

Microsoft had traced the sale of the suspect software to a client company's sysadmin. For purposes of this story, we'll call that sysadmin "Ed." When Linkous and other members of the Sabera team were secretly called in to investigate, they found that Ed had sold more than a half-million dollars in pirated Microsoft, Adobe and SAP software to his employer.

The investigators also noticed that network bandwidth use was abnormally high. "We thought there was some kind of network-based attack going on," says Linkous. They traced the activity to a server with more than 50,000 pornographic still images and more than 2,500 videos, according to Linkous.