Making trends transparent across languages

There’s a data revolution afoot, and both business and government are in the thick of it. In corporate IT, hardly a day goes by where you don’t hear about analytics. For the first time ever, executives can make decisions based on trends that are backed up by millions of data sources. Analytics take the guesswork out of business, and that’s a big deal.

The government uses analytics in a similar way. Agencies collect and analyze global information to study trends that affect intelligence, economics and health policies, among others. Analytics, it turns out, are just as important in the public sector as they are in the enterprise.

Analytics and Assimilation

So far, analytics have played a relatively limited role in the dissemination of information—though this will likely change quickly. But information assimilation by the public sector is a different story (please see further explanation of dissemination and assimilation in my last post). Just like businesses, the government uses analytics to find out more about what people are saying about choice topics that can then inform action and policy. Analyzing information from social media sites, online forums, videos and other sources is a key way to gather intelligence and conduct research. With analytics being used to track things like trending social media communications and news reactions by government today, there is an important push towards analyzing trending information in languages other than English, which can provide important clues to international movements and sentiment. Without this, there is less ability to make use of the social media analytics and data coming in from abroad.

A Lesson from the Arab Spring

The recent string of revolutions in the Middle East really impressed the importance of monitoring and understanding online content, especially that from unstructured sources like video and social media. Tunisian fruit vendor Mohamed Bouazizi committed suicide in 2010, lighting the powder keg that set off the entire Middle East. The citizens of Tunisia quickly reacted, and using social media, launched a resistance movement protesting political and social issues of their country. Tunisians were discussing the Bouazizi protest online, circulating footage of demonstrations and organizing future protests. A similar trend happened when the citizens of Egypt overthrew President Hosni Mubarak. One of the largest Egyptian protests had a Facebook page with more than 90,000 people signed up, according to the New York Times.

According to the Washington Post, after a Senate hearing in February of last year, Senator Dianne Feinstein indicated a need to focus more on “open-source intelligence.” In the intelligence community, this refers to publicly available sources of information which can include Facebook, Twitter, YouTube and other social media, but also radio, TV, newspapers, and the Internet (the term is unrelated to open-source software). Although protests were already taking place online, the West was less aware of the revolutions in progress until they were actually physically taking place. The U.S. government understands the importance of tuning into trending social media and continues to focus a great deal on analyzing all data, including unstructured sources like social media posts. 

Trends Tie Everything Together

Analysis is easy to talk about, but harder to execute. There are thousands of languages in the world and millions of sources of data. In the world of social media, people tend to use colloquialisms and linguistic shorthand, adding a translation challenge. Simply capturing data isn’t enough. It has to be translated, understood, and presented in a way that provides value.

By understanding trending information or any topic that starts to be hot the government can have its finger on the pulse of global movements and trends in many areas. We can better understand how people are being impacted by economic and foreign policies and in general better comprehend how policy affects people. We can better grasp the spread of disease and global health trends, as when people tweet about “having the sniffles today” or being affected by more major health epidemics. Disaster relief efforts can be significantly enhanced by a better understanding of what is happening, where, through an analysis of trending social media data. We can even measure varying responses across the globe to major events like the U.S. election.

With the entire world networked online, the public sector can’t afford to ignore any information in any location. Moving forward we will continue to see government use of more advanced analytics of real time information in order to inform policy. Analytics won’t just be a perk going forwards. They’re going to become an essential asset.

Copyright © 2012 IDG Communications, Inc.

It’s time to break the ChatGPT habit
Shop Tech Products at Amazon