This article is excerpted from The Art of Software Security Testing: Identifying Software Security Flaws, by Chris Wysopal, Lucas Nelson, Dino Dai Zovi, and Elfriede Dustin, by permission of Pearson Education. All rights reserved. Chapter 5: Shades of Analysis: White, Gray, and Black Box Testing
The Art of Software Security Testing: Identifying Software Security Flaws, by Chris Wysopal, Lucas Nelson, Dino Dai Zovi, and Elfriede Dustin, by permission of Pearson Education.
This chapter describes the different analysis techniques you can use during the security testing process. Different situations require different methods of analysis. We also describe the different lab environments that can be set up to perform different types of testing.
White box testing is common in the quality assurance world. It is sometimes called clear box, open box, or simply informed testing. In white box testing, all information about the system under test is known to the tester. In the security world, this can also be thought of as an insider attack. The tester has access to the source code and design documentation. This allows the tester to be efficient. He can threat-model the system or do a line-by-line code review, looking for information to guide the selection of test data.
White box testing is the most efficient way to find security vulnerabilities. Why hide information from the security test team? More information allows quicker and more complete generation of interfaces to test. It also gives you an accurate picture of the system's security because it doesn't rely on security by obscurity, which is the hope that attackers will never discover information about how a system works. Security by obscurity is not real security. You should always assume that eventually all information about a system will be discovered or leaked. A well-designed and well-implemented system will still be secure. This is why good crypto algorithms can be published for review. They don't rely on privacy for security.
Chapter 4, "Risk-Based Security Testing: Prioritizing Security Testing with Threat Modeling," described a process for working with architects and component owners to determine the software's design. That process should be followed before any white box testing takes place. Doing so will uncover the software's attack surface and will show you what functionality was put in place to mitigate the risks that the attack surface poses. White box testing tests these mitigations.
A sample mitigation is the security mechanism put in place to counter the threat of inadequate randomness in a session identifier. After this mitigation is discovered through the threat-modeling process, the security tester can inspect the code that generates the session identifier. This usually is not enough because small mistakes that can be difficult to determine by inspecting the code can cause the randomness to be weak and vulnerable. A white box test would be to automate the process of creating a new session and to record the session identifiers that are generated. These identifiers can then be subjected to a mathematical analysis to see if they are truly random.
Black box testing involves examining the system as an outsider would, using tools to detect the attack surface and probe the system for internal information. With no internal knowledge of the system, the tester builds an understanding of the system. Information leakage is especially important to the black box tester because it helps him build more understanding than he would otherwise get by manipulating a leak-free program.
Many testers swear by black boxing techniques to complement white box testing. If too much emphasis is given to specifications and design documentation, the tester may miss parts of the system that were built incorrectly or were not included in the documentation. This out-of-spec functionality may harbor security flaws that must be discovered. Black box testing lets the tester probe all of the attack surface and generate test data for functionality that may not be in the design. A common mistake is not removing debug-only commands before production. Another is to throw in last-minute functionality that is not properly documented in the design. Black box testing can find flaws in these cases that would otherwise go unnoticed by the white box tester.
Black box testing can be used when a system is deliberately using security by obscurity to help protect information, as is often the case in digital rights management (DRM) systems. Software-only DRM is impossible to completely secure because the attacker has control of the system where the DRM software is executing. The best a DRM manufacturer can hope for is to raise the bar for a successful attack so high that a would-be attacker gives up. Black box testing by a skilled reverse-engineering team is often used to test the strength of the obscurity used. This typically requires expertise outside the norm for a quality assurance team and can be expensive. However, for DRM systems, it is necessary to demonstrate the efficacy of the obfuscation used.
Ideally both white box and black box techniques are used during security testing. White box testing is used to discover flaws in functionality that were specified in the design and development. Black box testing is used to discover flaws without having access to these application internals. Sometimes this combination is called gray box testing.
The application security tester typically performs gray box testing to find vulnerabilities in software. Flaws due to design and flaws due to unspecified functionality are equally important to discover. Because the source code is available to the security tester, it should be used to improve productivity.
Running the software under test in a debugger is the perfect way to meld a running black box test with the source code to give the tester the advantage of the gray box. In the Windows world, Microsoft Developer Studio is typically the debugger of choice if debug symbols and source code are available. It allows the tester to navigate easily through the stack and memory to explore complex variables such as classes and structures. In the UNIX world, gdb is typically used for this purpose.
As soon as the software is running in the debugger, the normal tools of black box testing can be brought to bear on the executing program. Fuzzers or automated regression suites are typically deployed for this. The tester can set break points on lines of code that are dangerous to see if they can be reached with external input to the program. These dangerous lines of code can be discovered with a code review or simply by greping or searching the code.
An example of a dangerous code is a sprintf statement in a C program that has a %s in the format string. If the source buffer that is copied to the destination buffer is too large, a buffer overflow condition occurs. But not every sprintf statement with this condition can be exploited. Gray box testing lets you find the lines of code that are truly exploitable.
?SomeFunction(char *input)
{
..
char dest[50];
..
sprintf(dest, "The
output is %s", input);
..
}
The preceding might be an exploitable problem, but sometimes you cannot easily determine from source code if that is true. The path user input takes to get to the input variable can be complex. A break point can be set on this and any other similar lines of code. Then the fuzzer or another automated test can be run to see if any of the break points are hit. If a break point is hit, the stack can be inspected to see what the control flow was to reach that point and if any validation is placed on the input. If there isn't, the test input can be manipulated to determine if the vulnerable line of code can be reached with data that causes an exploitable condition.
If issues are discovered during the development of the code, it is almost always a good idea to just fix the potentially exploitable code. But many development teams are loathe to fix issues that might be a problem in code that has already shipped to the customer or that was put into production due to the patching cost. A gray box approach to finding vulnerabilities gives critical exploitability information to the development team. This helps them make an informed decision about fixing a potential vulnerability in the code.
A dedicated lab with the appropriate hardware and software tools is essential for efficient security testing. You will undoubtedly build your own custom tools over time as you write custom clients or customize off the shelf tools. Many of these tools are open-source, so you can use them as a starting point for building customer clients.
VMware or other virtualization programs should be used to have standard installations of operating systems available. A file server can store images. This way, the OS can be brought to a known good state. It is possible through fuzzing and other automated testing to corrupt configuration files, the Windows Registry, or even system binaries. If this happens, you don't want to suffer through a full reinstall. A quick refresh with a saved VMware image can return you to the state you need to be in to continue testing.
The following sections list some of the important tools you should gather for your test lab.
Open-Source
SPIKE (fuzzer framework with some prebuild protocol fuzzers)
Peachfuzz (another fuzzer framework)
Mangle (HTML fuzzer)
FileFuzz (file fuzzer)
Commercial
beStorm
Codenomicon
Before writing your own fuzzer, check the fuzzing tools Wiki page to see if one already exists for the input type and protocol you are testing.
Open-Source
tcpdump
Ethereal (probably the only one you will need)
Snort (an intrusion detection system [IDS] that can be used as a sniffer)
Dsniff (a collection of tools that can sniff on a switched network)
Commercial
AeroPeek
EtherPeak
Agilent Network Analyzer
Network General Sniffer (the original packet sniffer)
Open-Source
gdb
Shareware
OllyDbg
Commercial
Microsoft Developer Studio
SoftIce
Obviously you need hardware that meets the requirements for the software to be tested. This may mean having Intel x86, PowerPC, or SPARC systems available.
Additional machines are necessary to run the tools required to drive the tests, monitor the network, and act as proxies. Intel hardware with a Linux distribution installed is usually the best solution. Most open-source tools are written for Linux. Some bootable distributions are targeted at the security tester. You can boot one of these CDs and turn a PC into a security test appliance. These include the following:
The telecom industry for many years has had commercial testing tools to test carrier-class telecom equipment and the implementation of the network protocols they use. An example is the Hammer products from Empirix for testing integrated services digital network (ISDN), Signaling System 7 (SS7), and voice over IP (VoIP) protocols. A few new companies are applying this approach to the standard Internet protocol suite. If the software you are testing implements standard protocols and your budget allows you to purchase tools, they are worth a look for outfitting your lab.
Your lab needs network hardware to perform network-based attacks and monitoring. A hub is ideal for connecting the test machine to the target and monitoring machines because it makes sniffing the network trivial. With a hub, all packets are forwarded to all ports so that each machine can see all the network traffic.
© Copyright Pearson Education. All rights reserved.
The Art of Software Security Testing: Identifying Software Security Flaws
By Wysopal, Chris/ Nelson, Lucas/ Dai Zovi, Dino/Dustin, Elfriede/
Published by Addison-Wesley Professional
Series: Symantec Press
ISBN: 0321304861; Published: 11/23/2006; Copyright 2007; Pages: Final; Edition: 1
www.awprofessional.com/title/0321304861