Companies offer services to crunch gov't raw data

IDG News Service - What if a U.S. president called for a bunch of government data to be released, but the raw numbers were difficult to make sense of?

A handful of companies and an open-source development project are trying to make sure that isn't happening as U.S. President Barack Obama pushes for open government in the early days of his administration.

The business models differ, but some companies are using the raw data released on Data.gov and elsewhere to demonstrate the power of their data-publishing and number-crunching services.

The release of all this data is a good move, but much of it is in a raw format, making it difficult to present it in a way that people can understand, said Kevin Merritt, CEO and founder of Socrata, a two-year-old company focused on helping government agencies and other users of the data reorganize and republish it on the Web.

Socrata calls its service of reorganizing the data into easy-to-read, interactive charts and graphs "social data discovery."

"The data is valuable, but the social data is valuable as well," said Merritt, a former Microsoft executive. "It's one thing to put the data online, but it's another thing to actually get some civic feedback loop."

On Data.gov alone, there were nearly 400 raw data sets available as of Wednesday morning.

There's a database of people, reported by country and region, granted asylum in the U.S. between 1998 and 2008; there's data on toxic chemicals released in Guam in 2005; there's a database of tornadoes, large hail and damaging wind reports from 1950 to 2006; there's data on the geochemistry of water samples in the U.S.; and there's a database of copper smelters around the world.

There's also data about patent applications, workplace fatalities, federal IT spending and migratory bird flyways. There are an additional 109,000 geographical data sets.

Vivek Kundra, federal CIO, was asked at a recent U.S. Federal Communications Commission forum about the Obama administration's philosophy on releasing data. "We don't really know which data feeds are going to lead to better analysis," he said. "What we are doing is, we're trying to release as much data as possible. As a result of that, we're finding a lot of innovation happening out there."

As the data is released, many U.S. residents are spotting trends that government workers hadn't seen before, he added. For example, based on data on which airline flights are typically late, fliers are starting to avoid flights from some airlines at certain times of the day, he said.

Socrata aims its products at government agencies, as well as journalists, researchers and other people wanting to make sense of the raw data. It offers a handful of products, including a free entry-level offering that allows anyone to host data on Socrata.com and a hosted, branded data site for large organizations. Socrata can help government agencies cut costs for storing and delivering data, Merritt said.