WWDC 2018 introduced significant enhancements to artificial intelligence (AI) on Apple’s platforms, so here’s a few ideas of what to expect.
Apple Maps (soon)
We recently learned that Apple’s fleet of Maps cars use LIDAR (Light Detection and Ranging) sensors when capturing data about the places they visit.
One easy way to understand what this technology does is iPhone X, which casts hundreds of tiny spots of infrared light on a user’s face in order to help drive the Face ID system.
That’s not a LIDAR system per se, but the tech works a little like this, capturing multiple data points about a place. This data helps the AI build a picture of the place, an image that should in theory enable on-board technologies to understand the difference between a lamp post and a pedestrian on a dark night.
My best guess is that in the future, you’ll never get lost, as you’ll point your iPhone camera at any location and Maps will compare the image with its own data to tell you where you are.
Magic Sudoku
Imagine AI that can take an image, understand the image and then compute the appropriate response to what it sees.
This is a few steps beyond a simple data-based interrogation/response machine intelligence model, as it demands a deeper, cognitive intelligence capable of responding correctly to challenges a programmer cannot predict.
Magic Sudoku is an app that solves Sudoku puzzles for you: Point your camera at an unsolved Sudoku puzzle, and it will figure out what you are looking at and subsequently provide the correct answers for you – in seconds.
This is a direct application of how AI/machine learning can augment your reality with tools you’ll use. After all, if you can solve Sudoku, what other problems can you resolve? At what point does AI begin to solve real-world problems, such as figuring out what’s wrong with mechanical devices? We know some big enterprises are already developing VR-based maintenance manuals and advice systems.
PupCam
Snapchat for dogs? I think so – and built in a matter of weeks in 2017 by developer Octavian Costache using TensorFlow, Core ML (with a little help), Swift, ARKit, SpriteKit and lots of work. PupCam is available in the App Store.
It’s not going to change the world by putting silly faces on pictures of animals, but it does show how Apple’s approach of providing developers with the tools they need to build quite complex machine intelligence projects will enable the creation of much deeper solutions to real world problems in time.
“Only two years ago, for similar technology (on human faces), SnapChat had to buy a company for $150 million dollars,” Costache writes. “Now iOS ships human face landmark detection for free.”
PupCam illustrates Apple’s rapid progress towards building and making available pre-trained machine learning models that developers can use to create real solutions.
In combination with its USDZ format that supports the creation of object and content management systems for AR, Apple is doing everything it can to create playgrounds that accelerate AI and AR development.
Polyword
Siri in iOS 12 will translate languages. Duolingo is consistently becoming highly effective for language teaching. But if you want to learn a new language, then you really should think about installing Polyword on your iPhone.
When you do, you’ll be able to point your camera at an object and the app will provide you with what that object is called in any one of 30 different languages.
This is a great (and friction-free) illustration of the power of computer vision and machine learning. It’s the same sort of combination that gives you those street sign recognition and translation apps that will inevitably one day be installed as standard in mapping and vehicle control systems. Your vehicle will know what speed it should be travelling at even if you don’t. Siri will already know what temperature you want your hotel room thermostat to set itself to.
Measure, Magic Plan, IKEA Place
Coming soon in iOS 12, Apple’s Measure app will let you precisely measure the distance between two points. That’s essential as the company seeks out ways to let people accurately share real-world objects in virtual space.
Apps such as Magic Plan and IKEA Place represent how such accuracy can be used constructively: Magic Plan lets you create floor plans, and IKEA Place lets you see what furniture might look like if put in your place.
Combine all three, and you can answer real-world challenges around interior design, object placement, and – perhaps the most challenging problem – ensuring you can get larger items of furniture into your home in the first place.
Replacing bureaucracy
Another example of machine intelligence combined with computer vision, Fyle lets you capture images of business receipts using the camera on your iPhone and then uses optical character recognition to extract the important data. The app will also create expenses reports in PDF, track vehicle mileage, track duplicates invoices, and more.
It isn’t too hard to see how automated accountancy systems could reliably handle such tasks. This begs the question: As machines become better at recognizing words and numbers and more capable of apprehending and acting out complex tasks, how many other traditionally white-collar jobs will become automated? Automation isn’t confined to Industry 4.0.
One more thing
What I’m saying isn’t a love letter to tech for the sake of it. It's just a simple attempt to articulate how rapidly AI development is accelerating to deliver real solutions to real problems.
Apple’s senior director of machine learning and AI, Carlos Guestrin, recently spoke at the CloudTech Summit. His fascinating presentation underlines just how profoundly these technologies are likely to change the status quo, which is why Apple and others must think so deeply about the ethics of what is being created and how it is used.
Google+? If you use social media and happen to be a Google+ user, why not join AppleHolic's Kool Aid Corner community and get involved with the conversation as we pursue the spirit of the New Model Apple?
Got a story? Please drop me a line via Twitter and let me know. I'd like it if you chose to follow me on Twitter so I can let you know about new articles I publish and reports I find.