NSA's alleged phone-records program puts spotlight on data mining

Computerworld - The controversy over the National Security Agency's terrorism-related surveillance efforts, including its purported program for collecting domestic telephone data, is shining a spotlight on the esoteric arena of high-end data mining.

One IT vendor that has been publicly linked to the NSA is Narus Inc., a Mountain View, Calif.-based company that sells systems for intercepting and analyzing telecommunications and network traffic. In an affidavit (download PDF) submitted in April as part of a lawsuit filed against AT&T Inc. by the Electronic Frontier Foundation (EFF), Mark Klein, a retired AT&T communications technician, said that in 2004, he saw a document listing Narus' technology among the equipment installed in a "secret room" at an AT&T central-office facility in San Francisco -- allegedly at the direction of an NSA agent.

The EFF filed a class-action lawsuit against AT&T in U.S. District Court in San Francisco on Jan. 31, claiming that the telecommunications carrier is violating federal law by letting the NSA wiretap its customers without warrants.

Steven Bannerman, vice president of marketing at Narus, declined to confirm or deny that his company is involved with the NSA and AT&T. But he readily acknowledged that its technology has the ability to sift through large amounts of network data in search of targeted information.

Narus' traffic processing engine can inspect data at speeds of up to 10Gbit/sec. while performing deep inspections of the content of network packets, including telephone calls, e-mail text and streaming video, Bannerman said. He claimed that the technology enables network operators to spot viruses and identify human targets, such as spammers or potential terrorists.

The equipment comes with optional lawful-intercept features designed to help ensure that only network packets presumed to originate from a court-approved target are tracked, and only for as long as a warrant is issued. But, Bannerman noted, "once we sell the product to customers, there's no mechanism in the software to check whether or not they are using the warrant management system."

The device that collects the packets is paired with an Intel-based "logic" server that runs Red Hat Linux and analyzes packets in real-time for preconfigured targets such as IP addresses or "voice prints," he said. It also can check for anomalous patterns.

Determining what patterns to scan for is done separately, typically by using data mining and business intelligence tools to analyze information stored in a data warehouse.

Stephen Brobst, chief technology officer at Teradata in Dayton, Ohio, declined to comment on whether the NSA is using the NCR Corp. division's data warehousing software. But he acknowledged that Teradata's technology is popular with telecom carriers and network services providers for storing and analyzing the massive volumes of call data records and network traffic information they collect.