How one firm turned government data into a business

WASHINGTON -- Through its data.gov effort, the White House wants to encourage more private sector use of government data. But turning this data into something useful is not necessarily going to be easy for either the government or the private sector, says one start-up, Panjiva Inc.

Each day, New York-based Panjiva receives via FedEx a disk from the federal government that contains U.S. customs shipping data on it. It cost $100 for each disk.

The government collects data from companies shipping to the U.S. to track duties. From that data, Panjiva built a database of 20 million shipping records from about 700,000 companies. It uses the database to discover the firms that are sending goods into the U.S., who is receiving them, as well as the type of goods, how often and in what quantity. Panjiva then maps relationships between buyers and suppliers.

The data is "incredibly messy," said Josh Green, Panjiva's CEO. "Think about how many ways there are to misspell the name of a Chinese factory." This data isn't online and is in an arcane, nonstandard format, he said.

But what emerges from this data is revealing. For instance, when auto supplier Lear Corp., filed for bankruptcy on Tuesday, Panjiva posted a chart showing Lear's shipment volume falloff.

Green worked with his CTO, Jim Psota, an MIT doctoral student in electrical engineering and computer science, to develop the algorithms needed to clean up and organize the data. The firm, which first piloted its system in January 2008 and operates on venture capital funding, has since acquired some big-name clients, including The Home Depot.

The customs data that Panjiva uses isn't on data.gov, and the vast majority of government data is a long way from appearing on the site because it isn't in standard formats or easily accessible. Data.gov is mostly being used today to aggregate federal data that's already been available.

But there are indications that the White House is taking the extra step to put new sources of data online, including a decision this month post online the salaries of White House staff.

There are expectations that what is being released by the government will be put to use by developers in new ways. Sunlight Foundation, a Washington-based group that advocates for open government, is running a contest through this summer that that invites developers to submit applications.

Green said his company illustrated how useful government data can be, but the usefulness of data.gov will be determined by the extent of the federal government's effort to release data.

"I think the most promising thing about data.gov is the tone it is setting," Green said. But the proof will be "how deep it goes into the various departments."

Forester Research Inc. analyst Christine Ferrusi Ros, said Panjiva's data benefits even firms diligent in investigating overseas suppliers. Over the course of a contract, some companies don't keep up with suppliers consistently, she said. "I think there are still a lot of risks for companies that go global that they don't know about it."