Language analysis software aids U.S. Web search for terrorist activity

Todd Weiss   Today’s Top Stories    or  Other Data Mining Stories  

Transferring Best Practice Analytics from Experts to Everyone When The BI You Know Isn't The BI You Need Gaining Insights Through Analytics Sign up to receive Security Resource Alerts

March 10, 2003 (Computerworld) -- It's one thing to track and monitor terrorists the U.S. government already knows about. But it's even tougher uncovering the ones who are unknown.
To help in that effort, a Cambridge, Mass.-based globalization software company, Basis Technology Inc., has created the Rosette Arabic Language Analyzer. The tool can plug into data mining applications used by U.S. defense and security agencies that are involved in scouring the Internet for Web sites written in Arabic. By automating the search, information that can help investigators find new potential suspects in the fight against terrorism can be gleaned quickly, according to the company.
Carl Hoffman, CEO of Basis Technology, said the analyzer, which is currently in beta testing, plugs into content management and knowledge management applications used by the U.S. government and defense contractors, including Convera's RetrievalWare and Fast Search & Transfer ASA's Data Search software. It's also available for Microsoft SQL Server and Oracle Text/interMedia. The tool allows the government to focus on obscure Web sites whose URLs change regularly, so the sites can be monitored for terrorist activity.
When interesting or worrisome information is found, it can be turned over to intelligence organizations for direct investigation by personnel who specialize in such probes, Hoffman said.
The analyzer is one of 37 Rosette foreign language analyzers offered by Basic Technology. The analyzers identify the language of the content and then convert the text into standardized Unicode, the international character set that provides a unique number for every character in any language. This is the first commercially available Arabic language analyzer created in the U.S., Hoffman said.
Basic Technology began working on the Arabic Language Analyzer shortly after the Sept. 11 terrorist attacks in the U.S., Hoffman said. "A number of government agencies in the intelligence community strongly encouraged us to move in this direction," he said.
Everette Jordan, director of the National Virtual Translation Center, an organization jointly sponsored by the FBI and CIA under the USA Patriot Act, said in a statement that "linguistics technology is beginning to play an increasingly important role when it comes to ensuring national security."
"Because of the enormous volume of multilingual intelligence information that must be analyzed with limited human resources, technologies that can assist in sifting, sorting and finding critical information are essential in ensuring that threats are detected as quickly as possible," Jordan said.
Glenn Nordin, assistant director of language intelligence policy at the U.S. Department of Defense, said in a statement that analyzers such as this one help because "U.S. government computer systems are largely designed to work with the Latin alphabet and U.S. character sets, [and] processing information in Arabic is a difficult undertaking.
"In the absence of universal transliteration standards, human transcript of foreign text into the Latin alphabet can result in significant corruption of the data and mismatches in searches," Nordin said. "Finding solutions that enable intelligence analysts to extract and disseminate information in the original language and script could be of critical importance."
Nordin and Jordan could not be reached for additional comment today.

Print this Story

Send Us Feedback

E-mail this Story

Digg this Story

Slashdot this Story

"We don't need al-Qaeda to blow us up. We are perfectly capable of lighting the fuse ourselves, courtesy of our..." Read more... "Analyzing data from online and your network may be a little easier because of a new browser. Yes, a browser...." Read more... Read more Business Intelligence posts or See all Blogs

Hackers hijack a half-million sites in latest attack HP in talks to buy EDS for up to $13B Microsoft faults OEMs for some XP SP3 endless reboots More top stories...

Mozilla slates Firefox 3.0 RC1 for late May IPhone out of stock 'companywide,' say Apple sales reps Cisco data center chief steps down

Your help desk career: Dead end or launching pad? A role on an IT help desk is what you make of it, tech pros say — just don't get too comfy. The darker side of Webmail Web-based e-mail may be exposing you to privacy and security dangers you didn't sign up for. Performance showdown: Flash drives versus hard disk drives Ever been tempted to replace the mechanical hard drive in your laptop with a shiny new solid-state disk? Our expert did so, and here's what he found. Xerox touts erasable paper, smart documents PARC showed erasable paper and other technologies that adds intelligence to documents with raw text. Windows Vista A to Z Reviews, analyses, how-tos, visual tours, hot issues and predictions about Microsoft's new OS. More Continuing Coverage After 9/11: Homeland SecurityAsk a Premier 100 IT LeaderBlackBerry Legal BattleData Security BreachesElectronic VotingFirefoxH-1B VisasHP's Boardroom ScandalHandheld DevicesHurricane Katrina AftermathMicrosoft Legal IssuesMicrosoft's VistaOpen SourceRFID TechnologySCO's Linux FightSarbanes-Oxley ActSpamVoice Over IPWi-FiWindows Meta File VulnerabilityWindows XP Service Pack 2 IT Careers 2010 Four years from now, the IT field will be a vastly different place. Will you be ready? More Special Reports Premier 100 IT Leaders 2007 Best in Class2006 Computerworld Horizon Awards2006 Premier 100 IT Leaders2006 Salary Survey2007 Best Places to Work in IT2007 IT Forecast2007 Premier 100 IT Leaders40 Years of ComputerworldASPs, Take TwoBest Places to Work in IT 2003Best Places to Work in IT 2004Best Places to Work in IT 2005Best Places to Work in IT 2006Business Intelligence Home RunsBusiness Intelligence: Smarter BIBusiness Intelligence: The Future of BICRM Goes VerticalCRM: Sober CRMCareer Planning Guide 2005Careers: IT Profession 2010Computerworld Horizon Awards 2005Data Management: Mining for GemsData Management: Taming Data ChaosDevelopment: Hard-Workin' Web SitesDevelopment: New Tools New ChoicesDevelopment: The New World of Application DevelopmentDevelopment: The Web Services TsunamiDevelopment: Web Services HurdlesDisaster Recovery: Preparing for the WorstE-commerce Grows UpE-mail/Groupware: Big DecisionsFaces of Mobile ITForecast 2006Global MobileHardware: Multiple Cores, Multiple ChallengesHardware: On-Demand Un-HypedHardware: The Shape of Things to ComeHorizon Awards: 10 Cool Cutting Edge TechnologiesIT Management: Guide to Managing VendorsIT Management: Navigating Global ITIT Management: Recovery AheadIT Management: The Future of ITIT Management: The Resourceful Project ManagerInnovative Technology Awards 2004Management: Reinventing ITMicrosoft in TransitionMobile/Wireless Leaders and LaggardsMobile/Wireless: The Untethered WorkerMobile/Wireless: Tiny Gadgets, Huge CostsNetwork Management: Taking ControlNetworking: Turbulence Ahead!Networking: VoIP Goes MainstreamOperating Systems: In the Slow LaneOperating Systems: Linux Goes GlobalOperating Systems: Should You Nix Unix?Outsourcing: Offshore Buyer's GuideOutsourcing: Outsourcing DangersPremier 100 IT Leaders 2004 Best in ClassPremier 100 IT Leaders 2005 Best in ClassROI: Do the Math!Salary Survey 2003Salary Survey 2004Salary Survey 2005Security Action PlanSecurity: Compliance HeadachesSecurity: Proactive SecuritySecurity: Risk and RewardSecurity: Tips From Security ProsSouped-up SecurityStorage: Battling ComplexityStorage: Cheap & Secure Data StoresStorage: Stretching Your Storage DollarsStorage: The Lean Storage MachineStorage: The New Rules of StorageStorage: The Ultimate Backup GuideSupply Chain: Missing LinksSupply Chain: RFID Reality CheckThe Business of SecurityWeb 2.0 SecurityWireless: Wireless At Work All Zones Application Performance Zone Enterprise-Class Security Zone Enterprise Solutions Zone The File Data Management Zone Grid Computing on Windows Zone Security Management Zone ITIL Best Practices Zone The SAS Zone Storage Virtualization Zone The Data Center Management Zone See your link here

Computerworld Report: Virtual Reality Download this Computerworld Report, free for a limited time, compliments of HP. (Source: Computerworld) The data center is real, but storage is turning virtual at many organizations that need to manage exploding storage needs. Learn how virtualizing your enterprise will save you money in this Computerworld Report, a $49.95 value, available free for a limited time, compliments of HP. Download this executive briefing  Managing B2B Web Channels For Success Download this survey results white paper for free, compliments of Tealeaf. (Source: Tealeaf) In March 2008, Computerworld invited Web site visitors to participate in a survey regarding their organizations' use of the Web. Specifically, the goal of the survey was to better understand organizations' goals and priorities for their B2B Web channels in 2008. The survey was commissioned by Tealeaf, but data was gathered and tabulated independently by Computerworld Research. The following report represents top-line results of the survey conducted among 250 respondents at organizations that leverage the Web for B2B commerce or self-service. Download this white paper  Long Tail Supplier Collaboration - What's In It For You? Long Tail Supplier Collaboration - What's In It For You? Download this webcast, free, compliments of Sterling Commerce Go to the webcast  White Papers Read up on the latest ideas and technologies from companies that sell hardware, software and services. New Fujitsu High-End Itanium Windows- and Linux-Based PRIMEQUEST Servers Offer the Utmost in High Availability New Fujitsu High-End Itanium-Based PRIMEQUEST Servers Offer Industry-Leading System Management for Linux and Windows Symantec State of the Data Center Report 2007 View more whitepapers

• Hackers hijack a half-million sites in latest attack
• HP in talks to buy EDS for up to $13B
• Microsoft faults OEMs for some XP SP3 endless reboots
• More top stories 

The Impact of Messaging and Web Threats Messaging, internal and Web-based threats are increasing in number and severity. The risks to organizations large and small are real problems that users and their employers face if they do not establish adequate defenses against this growing variety of threats. Read this Osterman Research paper to learn how organizations must implement a layered defensive strategy to protect against all types of threats and how Sunbelt Software can help. Download this white paper now


The Information Management Roadmap


Structured data. Unstructured data. Semi-structured data. It doesn't matter what kind of data it is or where it comes from. Your job is to develop a strategy around Information Management that harnesses any data needed to make business decisions, and to deliver it increasingly in real time. The success of your organization hangs in the balance.
Join Computerworld, SAS, and IDC to learn the best Information Management strategy for your organization.


Watch this webcast now