Big Data causes concern and Big Confusion

CIO - Big data gets a lot of buzz these days and organizations are increasingly concerned about the problem of managing it, but many don't really understand what big data is. Nor do they have the tools in place to effectively manage much of the data already at their disposal, says Mandeep Khera, chief marketing officer of LogLogic, which specializes in a scalable log and security intelligence platform (LSIP) for the enterprise and cloud.

"Most of them are concerned about big data, yet they don't understand what it means," Khera says. "Because there's been so much said about big data, there's no clear definition and everyone is confused."

A new survey conducted by LogLogic in conjunction with IT security research consultancy Echelon One finds that 49 percent of organizations are somewhat or very concerned about managing big data, but 38 percent don't understand what big data is and a further 27 percent say they have a partial understanding. Additionally, the survey found that 59 percent of organizations lack the tools required to manage data from their IT systems, instead turning to separate and disparate systems or even spreadsheets.

"We know that data is important from a lot of different perspectives: security, IT operations, compliance," Khera says. "Companies need to be managing data much more effectively so they can make more intelligent decisions."

The global survey was based on the responses of 207 individuals at director level and above in a variety of industries, including manufacturing, education, government, finance, healthcare, transportation, media and publishing and others.

"Big data is about many terabytes of unstructured data," Khera explains. "Information is power, and big data, if managed properly, can provide a ton of insight to help deal with security, operational and compliance issues. Organizations of every size are collecting more data from a variety of sources within the enterprise and cloud infrastructures, and many organizations are not using the right tools and processes to manage these data. If this pattern continues, we will see enterprises falling further behind, unable to derive actionable insights which can help organizations make intelligent decisions."

Most respondents to the survey -- 62 percent -- said they already manage more than one terabyte of data. But more is coming. The volume of data in the world is increasing at a nearly incomprehensible rate. IBM says we create 2.5 quintillion bytes of data every day. And perhaps more astonishing, 90 percent of the data in the world today was created in the past two years according to Big Blue. The data is coming from sensors, transaction records, images and videos, social media posts, logs and all sorts of other sources.