Sentiment analysis comes of age
Figuring out how a company or product is viewed by Netizens has gotten easier
Computerworld - Sentiment analysis is not a new concept. Business, marketing and investment strategists have long been scanning newspapers, industry publications and the like to scope out how the public, consumers and influential pundits view a particular company, brand or product.
Two recent developments are dramatically changing the game, however. First, the Web has seen a rapid proliferation of online news services, blogs and other social media that post up-to-the-minute information and commentary about companies and their products. Second, semantic-software vendors -- including Lexalytics, Expert System, Endeca, Cambridge Semantics and Cymfony -- now offer sentiment analysis tools as components of their platforms.
In fact, a growing number of semantic platforms can query and analyze material from Web 2.0 sources such as blogs, Twitter, and social networks like Facebook and LinkedIn. (For more about semantic technology, see our main story.)
Other vendors that offer software or services that handle sentiment and semantic analysis of Web and Web 2.0-sourced material include Dachis Group, Evolve24, Radian6, OpenAmplify, DNA 13 and recent start-up AdmantX.
Sentiment analysis platforms use two main methodologies. One involves a statistical or model-based approach wherein the system learns to assess sentiment by analyzing large quantities of pre-scored material. The other method utilizes a large dictionary of pre-scored phrases.
With either option, the customer often needs to add the scoring for domain- or specialty-specific phases.
Dow Jones, in collaboration with Columbia University and the University of Notre Dame, has compiled a dictionary of about 3,700 words that can signal changes in sentiment. Positive words include ingenuity, strength and winner; negative ones include litigious, colludes and risk.
The dictionary is the basis for Dow Jones Lexicon, a service that allows securities traders and analysts to determine sentiment, frequency and other relevant complex patterns within news coverage to develop predictive trading strategies. In addition, Dow Jones has developed a tool that it calls the Economic Sentiment Indicator, which performs sentiment analysis on material from 15 daily newspapers to track and predict the health of the U.S. economy.
Getting to nuances of opinion
The marriage of semantic and sentiment analysis can provide a powerful tool for determining the import and meaning of a piece of information, industry sources agree.
Say a Ford marketing executive wants to check out current consumer attitudes toward a particular car model. A platform such as Expert System's Cogito can semantically query various industry blogs and postings and come back with potentially relevant comments such as: "The seats of my ugly Ford Explorer are great." Using a combination of semantic and sentiment analysis, it can then determine that "this is a positive judgment of seats, but not of the brand," says Expert System Partner Luca Scagliarini.
Another tool, Thomson Reuters' Machine Readable News service, is designed to assess the positive or negative impact of news reports on industries and individual companies, says Rich Brown, the service's global business manager. For example, a major hurricane threatening the Gulf Coast could benefit energy companies by pushing up oil prices and the prices of energy companies' stocks, but it could hurt companies that have rigs in the Gulf of Mexico.
The potential paybacks of using semantic and sentiment analysis tools and services are impressive. Deutsche Bank recently conducted a study to determine the ROI of using a service like Thomson Reuters' Machine Readable News to pick stocks. It found that picking stocks for a portfolio based on the results of news sentiment analysis generated yearly returns of 5% for a low-risk strategy and 12% for a high-risk strategy, after trading costs. The report concluded that "News sentiment is a promising alpha signal for quantitative investors.... because it opens up a whole new data source -- textual data -- that was previously impossible to use in a systematic fashion."
Still, analyzing Web content can be tricky, particularly when the material being analyzed includes emotional commentary peppered with "profanity, acronyms, sarcasms, multiple exclamation points" and the like, says Brown. Also, industry experts (and common sense) suggest that you'll get better results if you stick with well-known and credible sources and disregard postings by Joe Schmo Blogger who just loves to spout.
Horwitt, a freelance reporter and former Computerworld senior editor, is based in Waban, Mass. Contact her at firstname.lastname@example.org.
Read more about Applications in Computerworld's Applications Topic Center.
- 15 Non-Certified IT Skills Growing in Demand
- How 19 Tech Titans Target Healthcare
- Twitter Suffering From Growing Pains (and Facebook Comparisons)
- Agile Comes to Data Integration
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- What Datapipe customers need to know about the new PCI DSS 3.0 compliance standard This handy quick reference outlines what PCI DSS 3.0 is, who needs to be compliant and how Alert Logic solutions address the new...
- The 12 PCI DSS 3.0 requirements addressed by Peer 1 Hosting This handy quick reference outlines the 12 PCI DSS 3.0 requirements, who needs to be compliant and how Alert Logic solutions address the...
- Defense Throughout the Vulnerability Life Cycle This whitepaper provides insight into how to leverage threat and log management technologies to protect your IT assets throughout their vulnerability life cycle.
- The Critical Role of Support in Your Enterprise Mobility Management Strategy Most business leaders underestimate the importance of tech support when they choose an EMM solution. Here's what to put on your checklist.
- Live Webcast Best Practices for the Hyperconverged Enterprise Network To the Age of Constant Connectivity and Information overload
- Live Webcast Unmasking the Differences between Consumer and Enterprise File Sync & Share The consumerization of IT combined with the rapid pace of the modern mobile workplace is forcing enterprise IT teams to evaluate file sync...
- Live Webcast Government Agency Webifies Outdated COBOL Applications Let this CTO tell you how his agency converted 1980s-era green screens into an e-filing portal for the 100,000 cases handled each year...
- The New Way to Work Knowledge Vault This Knowledge Vault focuses on how, in today's increasingly virtual world, it's more important than ever to engage deeply with employees, suppliers, partners,...
- Getting Ready for BlackBerry Enterprise Service 10.2 Find out how BlackBerry® Enterprise Service 10 helps organizations address the full spectrum of EMM challenges, while balancing the needs of both the... All Applications White Papers | Webcasts