Video search still a tough nut to crack

IDG News Service - While text-based search services such as Google's and Microsoft Bing now come pretty close to consistently serving up what users seek, video search services remain inexact at best, said video archiving experts who spoke on a panel at last week's WWW2010 conference.

Yet the panelists agreed that video searching techniques must improve exponentially if people are to use the growing amount of video footage now stored on the Internet and elsewhere.

"If the material is searchable, it will be useful to the public," said video archiving consultant Jackie Ubois, who moderated the panel during the conference held in Raleigh, North Carolina.

Hans Westerhof, director of the Images for the Future program for the Netherlands Institute for Sound and Vision, explained the urgency for developing better video search.

In 2005, the Institute started a program to digitize its vast video archive. About 280,000 hours of video and audio footage, including movies, television shows and news footage, will be digitized. About 100,000 hours of footage have already been converted, taking up 3 petabytes of storage space, and the archive is expected to grow to 14 petabytes by 2015.

The problem the Institute faces with all this video footage is making it easy to find. Many of the older source reels of film had little if any metadata, or descriptive data. Reels of old television programs, for instance, had just the barest amount of information, such as the title of the program and the date it was shown. No information was included about the content of the program.

"For the material to be useful, we need metadata," he said. The act of creating metadata should be automated wherever possible. "Traditional cataloging does not work at this scale," he said.

Right now, the Institute for Sound and Vision is looking at automated ways of extracting data from the video, using tools such as speech and image recognition.

But developing tools for automatically cataloging video is much harder than developing the tools used to tag text content for a variety of reasons.

Video, unlike text, can only be reduced to individual pixels, which offer no information about the video as a whole, said Paul Over, the project leader for a National Institute of Standards and Technology program to stimulate development of better video search. A block of text, on the other hand, can be reduced into a series of words, the definitions of which are known, and can be analyzed to give a greater summary of the whole document.

Video has "no correlate to the word," he said, making video harder to catalog.

"Video is not easy. It is hard to extract the structure," said Marko Grobelnik, the program manager for the VideoLectures.net service on online lectures. "We still struggle with basic problems like object recognition."