Both Facebook and Google have been working hard at using computers and algorithms to identify people in photos. They've gotten really good at it.
We still don't know what they'll do with that technology. To a large degree, it's up to us. But first, we have to understand what's possible.
Why Facebook doesn't need your face
Facebook is one of the leading organizations in the world developing facial-recognition algorithms. Facebook software can now identify people in photographs as well as people can. Facebook's DeepFace (no, I'm not kidding -- it's called DeepFace) can tell whether the subjects in two different photographs are the same person with 97% accuracy. That's even better than the FBI's own Next Generation Identification system.
DeepFace achieves this amazing feat by analyzing faces, turning them into 3D models, then making it possible to recognize the faces from angles and under lighting conditions that are different from those in other photos of the same person. The technology uses more than 120 million parameters, and a page on Facebook's research website explains that the company "trained it on the largest facial dataset to-date, an identity labeled dataset of four million facial images belonging to more than 4,000 identities."
But that's not enough for Facebook. It wants to be able to identify people even when their faces aren't showing. Toward that end, Facebook researchers are developing a system that looks at hairstyle, body shape, posture, clothing and so on.
Facebook can now recognize people whose faces aren't showing with 83% accuracy.
Tellingly, the company tried to avoid freaking people out with this research by developing the algorithm using Flickr pics, not Facebook photos.
While Facebook's ability to recognize people is astonishing, so is Google's.
How Google identifies you without identifying you
Everybody oohed and ahhed at Google I/O last month when Google demonstrated the search feature in its newly announced Google Photos offering.
In fact, we Google+ users have been enjoying this capability for years. Google Photo's search engine can not only tell the difference between cats and dogs, but also identify dog breeds and perform other search feats that seem impossible.
It will even find photos based on adjectives that could be used to describe the images they depict. For example, when I searched my own photo collection using the word delicious, it showed me hundreds of pictures I've taken of foods and beverages that were, in fact, delicious. (It also showed a picture I took during a safari in the Masai Mara in Kenya of a cheetah devouring a gazelle -- I guess that was delicious to the cheetah and Google software somehow knew that.)
Of course, the Google Photos search tool can find people when you search for them. In fact, when you go to the search bar and click to select it, you're immediately presented with three options: People, Places and Things. When you click the More link on the People option, it will show you a picture of every person you have ever photographed -- in order, beginning with the person pictured most frequently.
Click on any of those photos to get all the pictures of that person. When you do that, you'll notice something interesting: Google Photos will show you not only the pictures where the person's face is clearly visible, but also pictures in which the person's face is hardly visible at all.
But unlike Facebook's approach, all the faces that Google Photos search recognizes are visible; I haven't found photos where the person's back is turned.
It's also interesting to note that while Facebook's technology theoretically sounds more advanced, it is still in the research phase and has not been released, whereas Google's search tool is in its shipped product. And it’s already available to everyone free of charge.
Google isn't revealing details about how its photo search works, but it probably uses methods that are similar to Facebook’s.
One of the most interesting and under-reported features of Google Photos search is that Google has chosen to not associate pictures of people with their identity. For example, when I search for my son Kevin -- who is an active Google user, including a Google+ user -- Google Photos doesn’t associate my photos with his identity in its databases.