Apple's next innovation: 3D gesture-based computing puts the 'post' in Post-PC

Apple [AAPL] is likely to continue its exploration of next-generation user interfaces, and I'm beginning to think we may see some movement toward gesture-based controls in future products, with an Apple television and iMac being obvious candidates for such controls.

[ABOVE: This is the technology Apple is using in its Maps app and calling Flyover.]

Inside the machine

The company has filed a host of patents for 3D interfaces, interfaces offering real-seeming interactions with objects displayed on the screen.

Three patent examples;

Recent patents suggest these tangible-appearing interfaces could also extend to powerful virtual experiences when wearing video glasses.

To understand the potential here, imagine your Mac desktop to be a three-dimensional place. As a user you might actually experience the sensation of walking through that space, reaching for the files and folders you need, controlling them through a mix of spoken word and gesture controls.

The entire experience would be visible to you through your glasses. The logical outcome would see your computer become discreet to an extent far beyond anything in Minority Report.

A January published Apple patent for an iPhone interface postulated such a three dimensional experience. "Users would navigate the iPhone by picturing the display as an "imaginary camera viewfinder" offering a glimpse into a digital world. By moving the imaginary viewfinder around, the orientation of the iPhone would determine what is displayed to the user on the screen," reported AppleInsider.

The interesting point is that once you dispense with mouse and keyboard-based user interfaces, you arrive in a place in which the computer becomes even more discreet. In conjunction with video glasses you don't even need the display, just a live data connection to the computer itself -- another step toward Post-PC.

[ABOVE: Leap represents an entirely new way to interact with your computers. It's more accurate than a mouse, as reliable as a keyboard and more sensitive than a touchscreen. For the first time, you can control a computer in three dimensions with your natural hand and finger movements.]


This may sound far-fetched, but insanely great developers are already building solutions that take huge steps toward this. Leap Motion, for example, has built an incredibly compelling interface that enables you to interact with your computer using gestures. It's an exciting technology which now boasts Andy Miller, Apple's former iAds chief, as its President and COO.

"Officially unveiled on May 21, the Leap immediately caught the attention and imagination of consumers, developers and media around the globe, spawning nearly overwhelming numbers of pre-orders and developer kit requests from over 150 countries," the company press release said of this new hire.

Naturally the introduction of the technology has excited speculation Apple may or should acquire the firm, if only to grab the technology before its competitors do.

Leap Motion meanwhile seems content to forge its own path. It offers a sensor device that introduces the tech to other computers (PCs and Macs) and is whispering of plans to offer its own app store through which independent developers can offer up their 3D-ready apps.

Maybe, maybe not

However, it might be that Apple has no need to pick up Leap Motion -- after all, the surfeit of 3D interface patents may well suggest the company has its own plans for this. It's also important to keep the Apple maxim in mind: it will only bring a technology to market if it is able to apply it properly.

Given that iOS 6 and Mountain Lion don't offer such features as yet, don't be surprised to see the company introduce them in small stages, perhaps beginning with 3D touch-based interfaces for its new Maps app. Also don't be surprised if, in time, you find the company always had bigger plans for the built-in Webcam inside most Macs than making Facetime calls.

[ABOVE: Another Leap Motion demo.]

Disruptive technology

Meanwhile industry observers at MIT Technology Review seem giddy with anticipation for the new interface, calling this the "most important technology since the smartphone", they write:

"The fact that the Leap can see almost any combination of objects - a pen, your fingers, all 10 fingers at once, should make every interface designer on the planet giddy with anticipation. If you thought that the touchscreen interface on the iPhone and subsequent tablets opened up a whole new way to interact with your device, imagine something that combines the intuitiveness of that experience with the possibility of such fine-grained control that you could do away with the trackpad or mouse entirely."

Apple's consistent approach to user interface innovation has always led the industry, and it's not afraid to license technologies (such as those it borrowed from Xerox for a fee) in order to deploy new UI ideas.

With this in mind it's impossible not to imagine a future connection between Apple and this exciting new start-up company. After all, while touch-based computing on a desktop or notebook seems an unsatisfactory and unnatural experience, gesture based interfaces seem eminently logical. Though we'll have to get used to our offices being populated by people who appear to be in a permanent Tai-Chi session.

Are you expecting gesture-based controls in future Macs? If so, what sort of scenarios can you imagine for it? If not, why not?

Got a story? Drop me a line via Twitter or in comments below and let me know. I'd like it if you chose to follow me on Twitter so I can let you know when these items are published here first on Computerworld.

Computerworld's IT Salary Survey 2017 results
Shop Tech Products at Amazon