Four students have hacked Apple's iOS 7s voice assistant to control third-party apps with Siri without jailbreaking their iPhone.
[ABOVE: Image c/o Les Chatfield/Flickr.]
This was an entry in the PennApps Spring 2014 hackathon, the largest university hackathon sponsored by Google (really?), Dropbox, and Facebook. The hack is called (perhaps ominously) GoogolPlex.
The University of Pennsylvania M&T program students, Ajay Patel, Alex Sands, Ben Hsu, and Gagan Gupta all use iPhone 5S. They claim third-party apps should be able to integrate themselves with their hack. However at present it only works on a proof of concept basis, as it requires a helper app be installed on your iPhone, and that app is unavailable on the App Store.
Explaining what the hack does, they said:
"GoogolPlex is a hack on Siri (iOS 7 compatible) that allows Siri to integrate with third-party apps and hardware with custom commands like “turn on the lights.” Setup takes a minute and no jailbreaking is required. GoogolPlex already allows you to play songs in Spotify…and control Philips Hue lights with simple voice commands, all through Siri."
Its called GoogolPlex for a reason, they say:
"We named our hack “GoogolPlex,” to symbolize the near-infinite number of possibilities Siri could reach with our hack. But there is also a strategic and necessary reason we named it this. When a user says “GoogolPlex, turn off the lights,” Siri hears this as “Google ‘Plex turn off the lights,’” interpreting that the user wants to Google something. Siri then searches Google for the words “Plex turn off the lights,” and this is where GoogolPlex comes in. We are able to get the words the user was asking Siri by intercepting the request between Siri and Google with a man-in-the-middle exploit."
Technical detail: Using a custom built proxy server the students made the phone connect to their own custom DNS server when Siri set off on a Google request. They point that request to their own server, which receives the Siri request. "Then, we can perform custom actions using third-party APIs to interact with various services and hardware."
Their server uses natural language processing methods to translate the command, if it’s directed at a supported application the server sends a response that launches the helper app on the iPhone to exercise the command. There's much more technical explanation on the student's own blog.
I've not yet considered the wider implications of the hack, but it seems to match the desire among iPhone users to use Siri to control apps and devices beyond the default.
"I feel the Apple platform at this moment is better for users and developers alike," Ajay Patel told me. "Although it is more restricted, it provides a superior experience for users in terms of app support and productivity. I think Google Now is a very interesting feature of Android, however."
It seems inevitable Apple will move to close the hack down, but I do hope it encourages the company to enable control of third party app and devices using Siri -- after all, CarPlay enables you to use Apple's technologies to control third-party car entertainment systems, so it's not like the company hasn't spent time thinking about it.
Developers may need to think about it, however.
"Opening a Siri API would mean developers would probably need a good understanding of natural language processing to integrate into Siri, as Siri would have to determine which third party app to launch," said Patel.
Got a story? Drop me a line via Twitter or in comments below and let me know. I'd like it if you chose to follow me on Twitter so I can let you know when fresh items are published here first on Computerworld.