Google publishes facial recognition patent, could use social network photos

If you are out in public, you are fair game, but how would you like it if a stranger took your picture and then ran a search to find out your name, online aliases and all the information about you via that image? We are very nearly there with automatic face-recognition technology and social media aggregation. Although several companies are exploring that realm, Google recently published a patent in Europe to use facial recognition and social networking combined to give visual search results.

privacy-visual-search.jpg

Google Goggles is an image recognition search for mobile phones which allows curious users to take a picture of anything, such as a landmark, and then run a search for info on the item in question.  A user  does not need to type or speak to start the search; instead it's as easy as opening the app, taking a photo and then waiting for the search results.

Although in December 2009, Google chose not to immediately start using facial recognition technology for Google Goggles visual searches, in August 2010 Google acquired Like.com which is a visual search engine currently used for shopping. Before Like.com assisted online shoppers, it was a visual search site called Riya which used facial recognition technology to search images that users had uploaded and tagged. Way back in 2005, Google almost purchased Riya. So by acquiring Like.com, Google also gained Riya's technology. Other patent filings have recently become public, suggesting Google has been working all along to apply visual query technology to searching for faces.

Google's recent patent filing for User Interface for Presenting Search Results for Multiple Regions of a Visual Query suggests that Google's visual search for smartphones may be evolving. When you snap a photo, it often includes many things like buildings in the background, street signs, or even people. When you query that image, Google may break the photo into all those different pieces and search for each object within the picture -- including facial recognition searches.

Another Google patent, Facial Recognition With Social Network Aiding, was recently published in Europe even though it was filed around the time Google bought Like.com. The patent abstract describes facial recognition searches for "one or more likely names" of the people in the image. After this visual query has "potentially" matched one or more personal identifiers for each person, it will scour "communications applications, social networking applications, calendar applications, and collaborative applications" to create a list of possible identities for each person. The patent description also mentions that if the photo is tagged with a person's name, "that picture might be used in future facial recognition queries to recognize the person."

In regards to privacy, Google has several possible scenarios: to send only one of the "identifiers" to the person searching; to possibly allow only the person identified to make the photo public, or to send a request after a person is positively identified, asking if the image can be a face search result for other people's visual queries from within their social network.

InformationWeek noted that the inventors listed on the patent application include "David Petrou, Andrew Rabinovich, and Adam Hartwig" who also worked on Google Goggles. It seems as though Google is close to reaching its goal of incorporating facial recognition into visual searches, but Google is certainly not the only company with big plans to utilize facial recognition searches.

The Interpreter suggested other applications like Viewdlw, with mobile facial recognition on the fly, might help "government stalkers." Viewdle allows users to tag and save "faceprints" of people, and "then share the faceprint of them with other Viewdle users so they can recognize that person too!" The photosharing privacy settings are integrated with Facebook privacy settings.

Some people may have no privacy issues with visual searches that use facial recognition, but there are plenty of people concerned about protecting privacy. For instance, when you leave your house and go out and about, do you wear a name tag out in public or at the mall as a constant visual aide for anyone and everyone to identify you? That seems a bit over the top to me, but isn't that similar to tagging a photo with a person's name and then that image becoming face search fodder? Wearing a name tag at all times in public is a personal choice, but there isn't much choice if someone else chooses to tag you in an image.

Dictators and governments are already identifying people based upon real names and photos posted in social media. CNN published an interview with examples of when the Internet might help a dictator such as Facebook requiring people to use their real name as opposed to pseudonyms. Another example was after the Iranian protests were over in 2009, the government went through Flickr and collected photos of protesters, published those pictures on government websites and then circled unknown faces in red ink.

Access Now is mostly concerned with Facebook's policy requiring real, full names and has launched on online petition called, "Unfriend the Dictators." Yet how much easier will it be to identify dissidents when facial recognition software allows a government to conduct face searches and compare photographed faces with their databases or even the millions of photos stored on Facebook or other social networking sites?

I'm not saying all facial recognition visual search technology is bad. It could be cool or it could be a stalkers dream come true. We all have good days and bad days, so the ability to utilize face search could be used accordingly. It could be used for something wonderful like love at first sight and helping track down that stranger first seen in a crowded public place. Or it could be used for some ill-tempered moment like snapping a pic to search and track down that jerk who cut you off in traffic. In the case of dissidents, face searches could be a death a sentence.

FREE Computerworld Insider Guide: IT Certification Study Tips
Join the discussion
Be the first to comment on this article. Our Commenting Policies