Big data to drive a surveillance society
Analysis of huge quantities of data will enable companies to learn our habits, activities
Computerworld - NEW YORK -- As real-time and batch analytics evolve using big data processing engines such as Hadoop, corporations will be able to track our activities, habits and locations with greater precision than we ever thought possible.
"It will change our existing notions of privacy. A surveillance society is not only inevitable, it's worse. It's irresistible," said Jeff Jonas, a distinguished engineer with IBM. Jonas spoke to a packed house of several hundred people Wednesday at the Structure Big Data 2011 conference here.
For businesses, the ability to determine where people are by using geo-locational data will help them personalize advertising and marketing materials disseminated via the Web. For example, if a company knows a customer is in Aruba, it won't bother showing him ads for restaurants in New York; it might market sunblock or scuba-diving excursions instead.
Knowing where people are will also enable companies to accurately determine which potential customer is which. For example, if there are five people in the U.S. who have the same name and the same date of birth but live in different cities, it would be possible to verify the identity of each individual by determining their locations at a given time.
"Just look at the last 10 years of address histories ... it is very telling if this is the same person or not," Jonas said. "Two different things cannot occupy the same space at the same time."
Jonas said 600 billion electronic transactions are created in the U.S. every day, and many of those transactions come from geo-locational data generated by cell phones, which through cellular towers, triangulate a person's exact location at any time. Wireless providers have that data in real time.
By looking at data over a period of years, corporations can know how you spend your time, where you work, and who you typically spend time with.
"This is super food [for big data analytics]," Jonas said. "With 87% certainty, I can tell you where you'll be next Thursday at 5:35 p.m."
"Big data" -- an industry term that refers to large data warehouses -- includes machine- and human-generated data such as computer system log files, financial services electronic transactions, Web search streams, e-mail metadata, search engine queries and social networking activity. In 2010 alone, 1.5 zetabytes of that kind of data was created, most of it machine-generated. Companies filled their data center storage systems with about 16 exabytes of that data last year, according to Jason Hoffman, founder and chief scientist at cloud software provider Joyent.
Bill McColl, CEO of analytics engine vendor Cloudscale, said that up until now, big data analytics has been about offline queries or "MapReduce" algorithms, which were developed by Google. But 90% of corporate data warehouse users say they want to move forward into a world with real-time analytics.
"Companies know if they can extract more insight from data faster than their competitors, they're going to win," McColl said.
Jim Baum, founder and CEO of Netezza, maker of a massively parallel processing (MPP) data warehouse appliance, agreed with McColl. Baum argued that if a corporate user has to wait even three days to get an answer to an analytics query, the user won't bother asking a follow-up question that could the key to unlocking the truly valuable insights the information has to offer.
"If I can get an answer in real time, I will ask the next question and the next question, and that'll be followed by another. Getting answers in near real time is critical. It's the enabler of what we can do with big data," said Baum, whose company was purchased by IBM last year. IBM's purchase of Netezza was among a flurry of big data analytics vendor acquisitions over the past year. Other deals included EMC's purchase of Greenplum, Hewlett-Packard's purchase of Vertica and Teradata's planned acquisition of Aster Data Systems.
- 15 Non-Certified IT Skills Growing in Demand
- How 19 Tech Titans Target Healthcare
- Twitter Suffering From Growing Pains (and Facebook Comparisons)
- Agile Comes to Data Integration
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- The value of smarter oil and gas fields With global energy requirements continuing to rise, the exploration, development and production of new oil and gas resources are shifting to increasingly challenging...
- Smarter Environmental Analytics Solutions: Offshore Oil and Gas Installations Example This IBM Redbooks® Solution Guide describes a solution for implementing smarter environmental monitoring and analytics for oil and gas industries. The solution implements...
- Piecing Together the Business Intelligence Puzzle Business intelligence (BI) technology collects and analyzes company data, delivering relevant information to corporate decision-makers in an effort to produce favorable outcomes.
- Harness IT -- An Introduction to Business Intelligence Solutions Learn the key selection criteria required to provide your organization with the capability to address structured data, unstructured data and mobile demands so...
- Live Webcast Increasing the Value of Your Reports and Dashboards Learn how incorporating other analytical capabilities such as predictive modeling and visualization can increase the value of your reports and dashboards by providing...
- The Software-Defined Data Center: Is your ADC ready? Data center transformation is accelerating beyond virtualization to next-generation cloud architectures and software-defined data centers, bringing new challenges for application performance, scalability and...
- Application Acceleration: Optimize the End-User Experience Watch this on-demand webcast and learn how you can optimize your web content, accelerate performance across any device and browser combination, and offload... All Business Intelligence/Analytics White Papers | Webcasts