Conducting "fire drills" is an important part of my job. I conduct various drills throughout the year to test our company's ability to react to different types of security-related events. I picked this week to set a fire and see how our staff responded. The reaction was one of surprise, but I'm pleased to say that the actions taken were exceptional.
The first question was what type of drill to conduct. Our fire drills must cover a variety of security issues.
 | | | This Week's Glossary TCP SYN Flood Attack: A TCP SYN Flood attack is a denial-of-service attack that takes advantage of the normal workings of TCP/IP, the transport protocol that's the foundation for the Internet and most LANs today. A normal TCP connection begins with a series of introductions. The client first sends a synchronization (SYN) packet to a server. The server then replies with a SYN packet accompanied by an acknowledgement (ACK) packet. When the client sees this SYN-ACK packet, it then sends back a final ACK packet (SYN-ACK-ACK), and the session begins. The purpose of the session could be Web page requests, e-mail retrieval, file transfer protocol or Telnet terminal sessions. In a TCP SYN Flood attack, the perpetrator sends the server a SYN packet from a spoofed (fake) IP address. The server then returns a SYN-ACK and waits for the final SYN-ACK-ACK packet, which never comes because the spoofed IP address doesn't exist. The server repeatedly waits, then retransmits the SYN-ACK packet to the spoofed IP address when no response comes. This process ties up server resources. The server reserves a finite amount of memory to hold SYN packets while waiting for these SYN-ACK-ACK packets from the client. In addition, the server's processor must generate the SYN-ACK packets and the network must accommodate the bandwidth necessary to send and receive packets. By flooding the server with spoofed SYN packets during a short period of time, the perpetrator can overwhelm the server. Memory fills up so that none is available for legitimate traffic, and the server hangs, crashes or reboots. |
| |||
Backups are a good example.
Every three months, I hold a drill in which I randomly pick a file, a set of files or an entire client company's database and have our administrators restore either the latest backup or a backup from an off-site archive to a specified location.
I have to do this because we have service-level agreements (SLA) that dictate certain service-level commitments to our customers. Our SLA regarding backups dictates that we will restore a company's data within a certain period of time or we must pay the company a remedy. The remedy might mean a free month of service or a cash payout. It all depends on the customer and level of failure to perform according to the SLA.
I also perform fire drills to test the reaction of our network operations analysts to security events. Typically, I use an event that produces some sort of indication on our intrusion detection system (IDS) sensors, which are configured to send an alert to a central monitoring station.
In the past, I usually did something like attempting to log in to a server as a root a bunch of times - like 50 times. That usually generates a lot of attention. I usually time the exercise and test to see if the analysts are able to trace the IP address to the source.
So what to choose this time? There are many tests that can be executed to test the security and responsiveness of people and infrastructure (meaning the IDS). But a recent e-mail I received from a Computerworld reader prompted me to test our infrastructure's resilience and response capabilities in relation to denial-of-service attacks.
Launching a denial-of-service attack is a very touchy subject because such an attack can cause servers to crash, hang or reboot. I wanted to make this a worthwhile test, one that would both challenge and set a sense of urgency for multidepartmental units.
For my testing, I planned to launch what's called a SYN Flood attack. For the uninitiated, I've written a brief definition of what a SYN Flood attack is and why these are so deadly. The attacks create a stream of requests that overwhelm a server's ability to respond, making the system inaccessible to legitimate users.
Having decided on the type of drill, the next question was the logistics of when, where and how to launch the attack. Should I set it in motion during normal business hours, or should I wake our engineers out of deep sleep? Which servers should I target? Do I take down legitimate customers and run the risk of crashing a database? If I do this and we lose customers because of it, my boss, the CIO, will be ticked off. Hmm, such decisions.
Choosing a Victim
I chose our corporate e-mail server as my victim. The attack won't directly affect customers, yet everyone will be screaming if they can't e-mail their buddies with the latest office gossip, jokes or links to cool Web sites. I had my security engineer build a Linux server and retrieve a good SYN Flood generator off the Internet. Guess what? There are literally dozens of denial-of-service tools available for free on the Internet. Some are just source code, which must be compiled, but some are the point-and-click Microsoft Windows versions, which can be installed and configured with ease. It's a scary world out there.
Releasing the Floodgates
I chose to launch the attack at 10:30 a.m., just after the morning meetings but before lunch. I ran a ping test against the e-mail server to watch for responsiveness as I began the attack. To launch the test, I ran a continuous stream of TCP SYN Flood packets against the server from a spoofed IP address. I picked the generic IP address 0.0.0.0 as the spoofed address. That way, I wouldn't risk picking a real IP address that might be vulnerable if someone on staff decided to retaliate.
Almost immediately after launching the attack, the e-mail server failed to respond. In fact, I later found out it had crashed - hard. About five minutes later, I got a call from the network engineering manager. He said that on one of the internal routers, he was experiencing an increase of traffic, which looked suspicious. I asked him to look into it and get back to me. A few minutes later, he advised that the packets seemed to be coming from a spoofed IP address and that they were half-opened connections - that is, SYN-ACK packets. I asked what could be done, and again he responded like clockwork: "Configure the Cisco router for TCP Intercept."
TCP Intercept is a Cisco router feature that actually intercepts the SYN-ACK packets and keeps track of them. If the router doesn't see the corresponding SYN-ACK-ACK back from the source (client) within a certain time period, the router breaks the connection on behalf of the server.
Like most companies, we don't run TCP Intercept all the time because it takes up quite a bit of the router's resources. We just use it as a response mechanism in the event that we become the victim of a SYN Flood event or attack.
After enabling TCP Intercept and rebooting the e-mail server, the fire drill was over. Yes, there was some postmortem work to be done, but the main goal of the fire drill was accomplished. The whole incident lasted about 15 minutes. I wished it could have been a little quicker, but nonetheless, it went well. At the end of the day, I was confident that our network engineering department knew how to recognize and handle a SYN Flood attack.
Backup recovery and denial-of-service testing are just a few of the many different types of fire drills that I can use to test the security posture of my company. My next test, which I am excited about, will be a social engineering test. This entails tricking someone into giving me unauthorized access.
I will accomplish this in two ways. First, I'll make a telephone call to our support desk in hopes of getting the support technician to reset or give me a password to our application. The second test will involve sending spoofed e-mail to gain access or exceed authority. I'll let you know what happens next time.