WWDC 2017

WWDC 2017: Everything you need to know about Apple and AR

In one move, Apple just made AR mainstream. It took years of planning.

Apple, iOS, WWDC, iPhone, iPad, AR, VR, iOS 11, ARKit
Credit: Jonny Evans

“We are high on AR for the long run, we think there’s great things for customers and a great commercial opportunity.” Apple CEO, Tim Cook, July 2016.

As I expected, Apple has introduced a powerful partnership and platform play to kick-start AR experience creation and distribution across its systems.

The company shared many of the details of its approach at WWDC 2017. This will have a big impact across consumer, creative, and enterprise markets so I thought it would be useful to put together what we know so far.

What is ARKit?

Apple has leveraged multiple technologies to create an AR ecosystem. When it comes to mobile devices, Apple has introduced ARKit, its new developer framework for the creation of engaging AR experiences.

The idea here is that consumers will use the cameras, processors and motion sensors already packed inside their iOS devices to access AR solutions. Your iPad or iPhone effectively becomes a window to an augmented world.

“By blending digital objects and information with the environment around you, ARKit takes apps beyond the screen, freeing them to interact with the real world in entirely new ways,” Apple said.

Apple on ARKit

Apple’s developer pages already provide a rich quantity of information around ARKit. They confirm (for example) that ARKit uses Visual Inertial Odometry (VIO) to accurately track the world around it using camera sensor and CoreMotion data. As I quietly predicted in May 2015, Apple has been able to build its implementation around world leading technologies it quietly acquired with Mataio.

Apple says ARKit provides:

  • Fast, stable motion tracking
  • Plane estimation with basic boundaries (horizontal planes at present).
  • Ambient lighting estimation
  • Scale estimation
  • Support for Unity, Unreal, SceneKit
  • Xcode app templates

Building for ARKit

Developers can build AR solutions using Apple’s state-of-the-art computer vision technologies.

Apple specifically suggests they will use ARKit to build games, immersive shopping experiences, for industrial design and more.

Here are a few examples of currently available VR solutions, but there is also big potential for AR deployments in logistics and distribution, emergency response, military, education, medical, and beyond.

Metal for AR

One of the more critical supporting technologies for Apple’s AR play is “Metal for VR” within macOS High Sierra. A component of the heavily improved Metal 2 system, Metal for VR offers a range of VR-focused features, including a VR-optimized display pipeline, viewport arrays and stereoscopic visualization frame debuggers.

Support for external graphics

Metal 2 also builds in support for external graphics processing units (GPUs). This means you can connect your MacBook or iMac to an external GPU system in order to make use of more powerful graphics processing, which is ideal for AR developers.

“Apps that use Metal, OpenCL, and OpenGL can now take advantage of the increased performance that external graphics processors can bring,” says Apple.

Don’t neglect that early versions of the Oculus Rift dev kit ran on Macs, but that effort failed as Macs couldn’t deliver the graphics support required. There was no news on Oculus at WWDC, but other partners were named.

Partnerships

I’ve said this before and I’ll say it again: ignore what you think you know and accept that Apple does partnerships. Its AR partners include Valve in order to provide SteamVR support on Macs. It is also working with Unity and Epic Games’ Unreal engine.

The Valve support means Apple has bought the HTC Vive into the fold – the company demonstrated some of the AR features using an HTV Vive headset.

“With partners like Apple, Google, Intel and dozens more, we will continue to make Vive the best VR platform for consumers, developers and enterprises alike,” said Vive.

On Unity support

It is in development, but right now Unity promises its plugin lets developers use ARKit features like world-tracking, live video rendering, plane estimation and updates, hit-testing API, ambient light estimation, and raw point cloud data.

“Radeon Pro 500 series graphics is recommended for 3D VR content,” said Unity.

Much more detail for developers here.

Steam VR for macOS

SteamVR for the Mac is now available in beta. It provides the same 360-degree, room-scale tracking as the Windows and Linux variants.

“On the development side, we have worked closely with Epic and Unity to make Mac extensions of content built on those engine technologies as simple as possible. Extension tools for those engines, and others, are available as part of this beta,” the company said.

Unreal Engine

Unreal Engine 4’s early access support for ARKit is available now on GitHub. Binary support is coming with the Unreal Engine 4.17 Preview in July followed by launch in early August.

And WebVR

Blink and you’ll have missed it, but WebVR will be supported in Safari on iOS and macOS. That’s significant as that technology should support Web-based VR experiences. Steam confirms it has been working with Mozilla to help enable WebVR support on Firefox, so macOS-based web developers can start using VR.

Where can I get it?

Developers running macOS High Sierra can make an immediate start on building VR apps using solutions from Unity and Valve. You can already download Unity’s Experimental macOS VR build here. You will also need Steam VR from Valve to use this, which is available here.

Return of the Mac

Apple knows the core Mac market.

At one time this market was defined by graphic designers, then musicians, web designers and video creatives joined the fold. The Mac felt the impact of mobile, while the next big tech wave, social, was fairly non-platform specific.

This changes with AR – Apple’s move to deliver a high-end Mac kit, such as the utterly droolworthy iMac Pro and (promised) future Mac Pro shows the company’s late realization that in order to create AR content it would need to also deliver an ecosystem which can be used to help create it.

Apple now wants to provide the world’s best platform for AR content creatives, from content creation to consumer use.

With this in mind it’s reasonable to expect a range of additional content creation messages from the company in the weeks ahead, look to iMovie, Final Cut X and discussion about new VR creation workflows around Apple solutions.

Massive deployment

We now know why Apple has made such a monumental, multi-year effort to ensure tens of millions of iPhone and iPad users are running devices equipped with seriously powerful processors.

That move to 64-bit everything was never trivial, it was a deliberate plan to ensure the company could deliver complex technologies, such as AR. Any mobile device with an A9 processor running iOS 11 will be able to take advantage of ARKit, which means any iPhone from iPhone 6s or later.

“With iOS 11, we’re delivering the biggest AR platform in the world, and it’s available today for developers to begin building AR experiences using ARKit for hundreds of millions of iPhone and iPad users,” said Craig Federighi, Apple’s senior vice president of Software Engineering.

This means developers working to create AR solutions will address a market of millions of potential users – and there will be big rewards available to developers who can create something viral.

Strong and stable

Just like UK Labour leader, Jeremy Corbyn, Apple is frequently underestimated by critics and enemies self-condemned to be forgotten by history. In this case, its move to AR has been well planned and seamlessly executed.

That should fill any enterprise with an interest in AR/VR with confidence. This tech can be part of a multi-year strategy for Apple, and for your company.

The VR Dev Kit

Apple has also begun selling its own VR development kit (which it calls the External Graphics Development Kit).

This is a Sonnet external graphics system with an AMD Radeon RX 580 8GB VR-ready graphics card, Thunderbolt 3 support and a Belkin USB-C to 4-port USB-A hub. Developers purchasing the $599 kit also receive a $100 which they can use to buy an HTC Vive VR headset.

Want to get started? Developers will need to download the Xcode 9 and iOS 11 betas, and then take a look at the ARKit documentation.

No Apple headset

I think Apple has been developing its own headsets for years. However, at this stage of the game it may well have decided that making nice with HTC and partners was more important to its plans than shipping Apple Glass. But I’m not convinced that it will be able to resist this wearable opportunity.

What people are saying

So, what’s the reaction to Apple’s AR plans? Here are a few notable statements:

“While we’re still very early in the maturity of AR, this is a good first step to get people thinking and playing with the technology,” Forrester vice president, analyst Michael Facemire told me.

 “AR is going to be the operating system of the future. The iPhone is going to be the AR device of the next five years, but eventually it's going to become some wearable," Munster said. "This ARKit will allow developers to build rich AR experiences far beyond what we've experienced in AR today." Loup Ventures, Gene Munster, CNBC.

“It’s a much more attractive platform for brands and businesses looking to create augmented experiences,” said Adam Fingerman, Chief Experience Officer and cofounder at SF-based ArcTouch.

“The digital world will overlay the real world in more detailed and accurate ways and you will be able to interact with Pokémon in a more immersive and life-like fashion,” said Niantic Labs.

“Apple’s emphatic entry into this space marks the beginning of AR and VR as a mainstream consumer phenomenon, as a market that’s now ready to grow beyond a few million enthusiasts, to reach hundreds of millions and then billions,” said Epic Games' Tim Sweeny in a blog post

This fall Apple will make AR mass market. You'd better start planning those Christmas and Thanksgiving AR virals straight away.

Got any more info? Let me know via Twitter so I can update this post.

Google+? If you use social media and happen to be a Google+ user, why not join AppleHolic's Kool Aid Corner community and join the conversation as we pursue the spirit of the New Model Apple?

Got a story? Drop me a line via Twitter below and let me know. I'd like it if you chose to follow me on Twitter so I can let you know when fresh items are published here first on Computerworld.

Computerworld's IT Salary Survey 2017 results
Shop Tech Products at Amazon