Language analysis software aids U.S. Web search for terrorist activity
Computerworld - It's one thing to track and monitor terrorists the U.S. government already knows about. But it's even tougher uncovering the ones who are unknown.
To help in that effort, a Cambridge, Mass.-based globalization software company, Basis Technology Inc., has created the Rosette Arabic Language Analyzer. The tool can plug into data mining applications used by U.S. defense and security agencies that are involved in scouring the Internet for Web sites written in Arabic. By automating the search, information that can help investigators find new potential suspects in the fight against terrorism can be gleaned quickly, according to the company.
Carl Hoffman, CEO of Basis Technology, said the analyzer, which is currently in beta testing, plugs into content management and knowledge management applications used by the U.S. government and defense contractors, including Convera's RetrievalWare and Fast Search & Transfer ASA's Data Search software. It's also available for Microsoft SQL Server and Oracle Text/interMedia. The tool allows the government to focus on obscure Web sites whose URLs change regularly, so the sites can be monitored for terrorist activity.
When interesting or worrisome information is found, it can be turned over to intelligence organizations for direct investigation by personnel who specialize in such probes, Hoffman said.
The analyzer is one of 37 Rosette foreign language analyzers offered by Basic Technology. The analyzers identify the language of the content and then convert the text into standardized Unicode, the international character set that provides a unique number for every character in any language. This is the first commercially available Arabic language analyzer created in the U.S., Hoffman said.
Basic Technology began working on the Arabic Language Analyzer shortly after the Sept. 11 terrorist attacks in the U.S., Hoffman said. "A number of government agencies in the intelligence community strongly encouraged us to move in this direction," he said.
Everette Jordan, director of the National Virtual Translation Center, an organization jointly sponsored by the FBI and CIA under the USA Patriot Act, said in a statement that "linguistics technology is beginning to play an increasingly important role when it comes to ensuring national security."
"Because of the enormous volume of multilingual intelligence information that must be analyzed with limited human resources, technologies that can assist in sifting, sorting and finding critical information are essential in ensuring that threats are detected as quickly as possible," Jordan said.
Glenn Nordin, assistant director of language intelligence policy at the U.S. Department of Defense, said in a statement that analyzers such as this one help because "U.S. government computer systems are largely designed to work with the Latinalphabet and U.S. character sets, [and] processing information in Arabic is a difficult undertaking.
"In the absence of universal transliteration standards, human transcript of foreign text into the Latin alphabet can result in significant corruption of the data and mismatches in searches," Nordin said. "Finding solutions that enable intelligence analysts to extract and disseminate information in the original language and script could be of critical importance."
Nordin and Jordan could not be reached for additional comment today.
Read more about Business Intelligence/Analytics in Computerworld's Business Intelligence/Analytics Topic Center.
- Data Visualization Techniques: From Basics to Big Data with SAS Visual Analytics This paper discusses some of the basic issues concerning data visualization, from data size and column composition, to solving unique challenges presented by...
- Best Practices in SAS Data Management for Big Data Big data trends and related technologies are becoming important to organizations of all types and sizes. This paper introduces the most important technologies...
- Fast and Furious: How SAS VA Helps IT Deliver BI Platform Read this whitepaper to learn more about the benefits of self-service BI to make business critical decisions.
- Understanding Big Data Quality for Maximum Information Usability In this paper we examine some of the challenges presented by managing the quality and governance of big data, and how those can...
- Cloud BI in Action: Recorded Webinar of Customer, Kony, Inc. See how Kony, Inc., a leading enterprise mobility company, is using TIBCO Jaspersoft for Amazon Web Services and Redshift to achieve embedded analytics...
- Cloud BI Overview: Jaspersoft for AWS Check out this overview of Jaspersoft for AWS, to easily and affordably build business intelligence solutions as well as embed visualizations and analytics... All Business Intelligence/Analytics White Papers | Webcasts
Our new bimonthly Internet of Things newsletter helps you keep pace with the rapidly evolving technologies, trends and developments related to the IoT. Subscribe now and stay up to date!