Snooping: It's not a crime, it's a feature

Computerworld - Cellphone users say they want more privacy, and app makers are listening.

No, they're not listening to user requests. They're literally listening to the sounds in your office, kitchen, living room and bedroom.

A new class of smartphone app has emerged that uses the microphone built into your phone as a covert listening device -- a "bug," in common parlance.

But according to app makers, it's not a bug. It's a feature!

The apps use ambient sounds to figure out what you're paying attention to. It's the next best thing to reading your mind.

Your phone is listening

The issue was brought to the world's attention recently on a podcast called This Week in Tech. Host Leo Laporte and his panel shocked listeners by unmasking three popular apps that activate your phone's microphone to collect sound patterns from inside your home, meeting, office or wherever you are.

The apps are Color, Shopkick and IntoNow, all of which activate the microphones in users' iPhone or Android devices in order to gather contextual information that provides some benefit to the user.

Color uses your iPhone's or Android phone's microphone to detect when people are in the same room. The data on ambient noise is combined with color and lighting information from the camera to figure out who's inside, who's outside, who's in one room, and who's in another, so the app can auto-generate spontaneous temporary social networks of people who are sharing the same experience.

Shopkick works on both iPhone and Android devices. One feature of the app is to reward users for simply walking into participating stores, which include Target, Best Buy, Macy's, American Eagle Outfitters, Sports Authority, Crate & Barrel and many others. Users don't have to press any button. Shopkick listens through your cellphone for inaudible sounds generated in the stores by a special device.

IntoNow is an iOS app that allows social networking during TV shows. The app listens with your iPhone or iPad to identify what you're watching. The company claims 2.6 million "broadcast airings" (TV shows or segments) in its database. A similar app created for fans of the TV show Grey's Anatomy uses your iPad's microphone to identify exactly where you are in the show, so it can display content relevant to specific scenes.

While IntoNow is based on the company's own SoundPrint technology, the Grey's Anatomy app is built on Nielsen's Media-Sync platform.

Obviously, the idea that app companies are eavesdropping on private moments creeps everybody out. But all these apps try to get around user revulsion by recording not actual sounds, but sound patterns, which are then uploaded to a server as data and compared with the patterns of other sounds.

Color compares sounds between users to figure out which users are listening to the same thing. Shopkick compares sounds to its database of unique inaudible patterns that identify each store. The SoundPrint- and Media-Sync-based apps compare sound patterns to their database of patterns mapped from all known TV shows.

Who else is listening?

Apps that listen have been around for years. One type of app uses your phone's microphone to identify music. Apps like Shazam and SoundHound can "name that tune" in a few seconds by simply "listening" to whatever song is playing in the room.

A class of alarm clock apps uses your phone's microphone to listen to you sleep. One example is the HappyWakeUp app. If you're sleeping like a log, the app avoids waking you. When HappyWakeUp hears you tossing and turning near the scheduled time, it wakes you up with an alarm.