As 'big data' grows, IT job roles, technology must change
IT managers say 'big data' movement continues to be a problem for many companies
Computerworld - LAS VEGAS -- As corporate data stores continue to grow, in some cases by more than 50% annually, the expanding task of managing and mining them for information is forcing a change in how IT workers are trained.
The discussion was sponsored by EMC's Isilon division, which manufactures clustered network-attached storage (NAS) systems used to house massive data warehouses under single domain-name spaces.
Chris McNally, a storage architect at IT hosting company Sungard, said he is helping to cross-train employees to learn how various systems fit into a larger IT ecosystem.
For example, McNally said, AIX and backup administrators have volunteered to undergo storage area network (SAN) and cloud storage training at Sungard.
"So instead of me being the storage guy who has to argue with these other guys about how this works and what we do, they're now able make intelligent requests with regard to storage," he said. "It creates a better product in the end."
James Lowey, director of network and computer systems at genome sequencer company Translational Genomics Institute (TGen), said workers in his company's traditional IT shop are required to learn how networks, operating systems and storage interact.
Lowey said mapping human genomes currently creates 2TB of new data every week, and he expects that to grow to 10TB per week by year's end. The genomic data is used to tailor drug compounds to treat diseases based on a person's specific genetic profile.
He noted that coming up with the best way to mine that data for important information continues to be a sticking point.
Looking to come up with a solution to that problem, EMC last year spent more than $3 billion to acquire companies like Isilon and data warehousing and analytics company Greenplum.
" 'Do I keep [data] or do I not keep it?' has been an age-old question," said Paul Rutherford, CTO of Isilon.
For Lowey, keeping all the data produced is an issue his company is struggling with. There's nothing more personal than genetic data, so keeping everything means keeping everything secure for as long as the company has it. But the data store also continues to be a valuable source of information that can be mined for creating custom drug treatments.
"The reason we keep everything forever is that we're not sure about what it is we have," Lowey said. "In life sciences, there's so much to learn and so much unknown."
EMC announced here that it has ramped up programs to train and certify "data scientists." A data scientist spends his or her time determining the value of a corporation's data.
Nick Mehta, CEO of cloud storage provider LiveOffice, said data persists whether it's properly stored or not.
"For us, the issue is, how do you enable a world where you can keep everything cost effectively? We want a way to keep everything and then make it valuable. Having all that data helps us do our jobs better," he said.
LiveOffice currently stores some 4 petabytes of data on disk and adds another 5TB to that pool each day. LiveOffice encrypts all of the data for customer safety.
- Data Warehouse Augmentation: The Queryable Data Store While organizations have, to date, been busy exploring and experimenting, they are now beginning to focus on using big data technologies to solve...
- Rebranded Quadmark revamps its IT solutions with Google Apps Switching to Google Apps halved Quadmark's IT admin costs while achieving 10% time savings per employee. The global consulting firm now spends 80%...
- CrashPlan PROe Security Because mobile laptops often are connected to unsecured networks, a very high standard of security is required to ensure privacy.
- Protecting Digitalized Assets in Healthcare Healthcare providers face an urgent, internal battle every day: security and compliance versus productivity and service. For most healthcare organizations, the fight is...
- Live Webcast Best Practices: How to Improve Business Continuity with Virtualization VMware solutions include a range of business continuity capabilities to help ensure availability for applications across your virtualized environment. Learn More>>
- Live Webcast
Transforming Finance, Procurement and Supply Chain Effectiveness with Cross-Functional Analytics
Date: May 6th, 2014
Time: 1 PM EDT
Attend this Webcast to find out how Oracle's packaged analytic applications enable line-of-business managers to examine all...
- Live Webcast LIVE EVENT: 5/7, The End of Data Protection As We Know It. Introducing a Next Generation Data Protection Architecture. Traditional backup is going away, but where does this leave end-users?
- LIVE EVENT: 5/7, The End of Data Protection As We Know It. Introducing a Next Generation Data Protection Architecture. Traditional backup is going away, but where does this leave end-users?
- Make or Break: New Auto Products Must Go To Market On Time This Webcast quantifies the value of time to market for the auto industry and highlights how Primavera Enterprise Portfolio Management can help organizations. All Data Storage White Papers | Webcasts