White House launches big data R&D effort

IDG News Service - Six U.S. government agencies will spend more than $200 million to help the government better organize and analyze large volumes of digital data, in a new "big data" research and development effort announced by President Barack Obama's administration Thursday.

The White House Office of Science and Technology Policy's (OSTP's) Big Data Research and Development Initiative will focus on building state-of the-art technologies to collect, store and manage huge quantities of data. OSTP wants to use the technology to accelerate discovery in science and engineering fields and improve national security and education, the White House said.

Among the new research is a Defense Advanced Research Projects Agency (DARPA) effort, which costs about $25 million a year, to develop methods to analyze large volumes of data, including unstructured data such as text documents and message traffic.

"In the same way that past federal investments in information-technology R&D led to dramatic advances in supercomputing and the creation of the Internet, the initiative we are launching today promises to transform our ability to use big data for scientific discovery, environmental and biomedical research, education, and national security," John Holdren, director of OSTP, said in a statement.

The IT industry has been buzzing about the use of big data in recent months, and the U.S. government collects and holds massive amounts of digital data. Universities and private companies can join the White House effort to explore the benefits of big data, Tom Kalil, OSTP's deputy director for policy, wrote in a blog post.

"We also want to challenge industry, research universities, and nonprofits to join with the administration to make the most of the opportunities created by Big Data," he wrote. "Clearly, the government can't do this on its own.A We need what the president calls an 'all hands on deck' effort."

Some companies are already sponsoring big data competitions, and universities are creating courses to train a new generation of data scientists, Kalil added.

Agencies involved in the new initiative include the National Science Foundation (NSF), the Department of Defense (DOD) and the Department of Energy, the White House said in a press release.

Among the new projects at the NSF are a $10 million project based at the University of California, Berkeley, that will explore three approaches for turning data into useable information, machine learning, cloud computing and crowd sourcing.

The NSF will also provide grants to support EarthCube, a project to allow geoscientists to access, analyze and share information about the planet.

The DOD will spend about $250 million a year, including $60 on new research projects related to big data. One DOD goal is to harness big data in ways that can lead to autonomous robotic systems.

The DOD will announce a series of big data prize competitions in coming months, the White House said.

Grant Gross covers technology and telecom policy in the U.S. government for The IDG News Service. Follow Grant on Twitter at GrantGross. Grant's e-mail address is grant_gross@idg.com.