Why (and how) to start planning your HoloLens apps

With Microsoft HoloLens already in the hands of the first developers, it’s time to start planning how your business can use augmented reality.

HoloLens
Credit: REUTERS/Beck Diefenbach

Augmented reality isn't new. Boeing has been using it since the 1990s to speed up installing the hundreds of miles of writing inside its planes. But those early systems were both clunky and expensive, and, so far, more portable augmented reality wearables like Google Glass haven't taken off. Microsoft's mixed reality HoloLens looks set to change that.

Microsoft announced last week that businesses can buy HoloLens now with the Microsoft HoloLens Commercial Suite. They get the Development Edition hardware as well as enterprise features for security and device management.

HoloLens has several advantages over virtual reality (VR) headsets. It's self-contained, so it doesn't need to be tethered to a PC. It lets you see the world around you (and other people that you're working with) through the 3D 'holograms' that it paints on top of the world. And you can develop for it using familiar tools like Unity.

In addition, several existing design tools will support HoloLens for viewing 3D designs, including Autodesk Fusion 360, Trimble and Dassault Systèmes. It also helps that Microsoft won't be the only hardware supplier. The company is sharing the technology behind HoloLens with some OEMs, so you'll be able to buy devices from several suppliers.

It's not too early to plan how your business could use HoloLens, suggests Laurens Frijters, CEO of Dutch consultancy Sparked, which has been working with the first HoloLens units available in Holland. "The next six months is a good time to start exploring, because in every new technology area it's the first few who are successful. Now is a good time to start your research and to look at how you can innovate in your business processes using mixed reality."

At its recent partner conference, Microsoft showed several custom HoloLens solutions, from the PGA Tour using it for golf course design, event planning and coaching to Japan Airlines, who plans to use HoloLens to train mechanics by letting them see inside jet engines while they're running.

Fritjers says the key areas where HoloLens will be immediately useful to businesses include design collaboration, virtual design and engineering, training and decision support. Getting access to remote experts using Skype and HoloLens will be useful for customer support and for collaborative work.

HoloLens at work

Sparked has already created a HoloLens system for Volker InfraDesign, a construction company that designs and develops huge infrastructure projects in the Netherlands and internationally. “We worked with them on how augmented reality can change the way they design and architect landscapes and infrastructure, and we built a proof of concept that shortened the lead time for design processes by giving architects and their clients and project managers access to an augmented reality where they could look at objects in 3D and walk through several scenarios," says Frijters.

Unlike a traditional 3D model, you can stand inside a HoloLens design of a building to get a real sense of what it will be like to walk through. In addition, the mixed reality can show moving objects like cars, planes and other people. And it can do that where the building will be.

Being able to see what the finished project will look like when you're on the actual site makes it much easier to decide between different options than looking at 2D plans and designs. Having discussions about how the design looks in situ before it's finished is much more efficient than having the same discussion after you've already spent time and money building it, which means projects can move faster and you’ll end up with happier customers.

Volker InfraDesign used the HoloLens system to plan where to place the public artwork that's being installed as part of improvements to the N18 highway in Holland. It allowed them to see how the artwork would look once it was in place, both from the road and from an overpass. But it's just as important from an engineering perspective to be able to see how the design fits into the location, to make sure that the mechanical design is correct and to spot any changes you’ll need to make before building starts.

This kind of design vision would be equally useful in retail, You could see different layouts for a store, complete with the shop fitting and even products on the shelves.

Sparked is also working on a virtual training system with Volker InfraDesign so they can expand their existing training centers (which already include VR training) to more locations. "We're looking at how to apply mixed reality in learning so you can communicate and collaborate and still have the scenario enhanced with virtual objects," Fritjers says.

HoloLens lets you see detailed information about objects you look at. "Integrating data into HoloLens from several data sources to give you more context and support fast decision making is the next step," he says. One customer needs to work with a lot of different pieces of art. "They want an instant overview of the context for each piece to make sure it's handled in the appropriate way. They need to see the metadata — who owns the artwork, what's the history of it, what procedures do they need to follow to move a piece of art, has it been damaged, is there a history of restoration that affects how it has to be handled or where it can be placed, what's the location it's going to… getting that information usually means searching through a web site or a file management system. We can integrate it together and show it to them while they're looking at the object, so they can make a faster decision by using HoloLens.”

Pumping the brakes

The opportunities for HoloLens are wide, but there are areas that may be off limits, for legal reasons — and for human ones. For example, some cities are considering HoloLens as a way of improving the service they can offer to residents, but the ability to recognize both objects and faces raises privacy concerns.

Virgin Atlantic tried out both Google Glass and smartwatches to help check-in staff at airports look up passenger information without having to turn to a computer. "It turns out that looking at your wrist is the universal sign for I'm bored with you," says CEO Craig Kreeger, explaining why the smartwatches proved unpopular. Glass worked better, but it’s more obtrusive. Seeing staff wearing something as large as HoloLens might also be off-putting for customers.

Sparked has one client who wants to use HoloLens for pilot training, but it's not clear if they'll be allowed to. "There are very strict regulations about wearing head mounted devices in a cockpit," Fritjers says.

Getting ready for HoloLens

Getting a HoloLens — or any mixed reality — project right will need a range of skills. “You need a domain expert, someone who knows how users do their work and how you can improve that work scenario. Then you need an expert who understands how you can use HoloLens and what kind of interactions you can offer those users. You need an expert designer and someone who can deliver the 3D models and content," says Fritjers. If you’re using the spatialized audio that makes a mixed reality environment feel more real, add an audio designer and sound engineer. "And you need a universal app developer" to bring it all together in a Universal Windows Platform (UWP) app.

Not all businesses have that mix of expertise in house, which is why you will want to start experimenting soon, or working with agencies. So far there are eight agencies in the U.S. and Canada who are part of Microsoft’s HoloLens Agency Readiness Program, working with customers like Skanska, Anheuser-Busch and the PGA Tour.

One such agency is Object Theory, which specializes in mixed reality applications for HoloLens (and Microsoft's wider Windows Holographic platform). In addition to working with a number of clients, it has come up with a Mixed Reality Collaboration service to help businesses build multi-user systems with life-size 3D avatars and 3D models that you can show and annotate to help teams collaborate from different locations. Think of it as a virtual conference room where people and objects look just as present whether they're right next to you or on the other side of the country.

That grew out of work for engineering and construction company CDM Smith — they wanted to be able to let people walk through a virtual 3D design on site, but without everyone having to be in the same place.

Lorraine Bardeen, General Manager, Windows and HoloLens Experience Production & Strategy at Microsoft, suggests some practical points for integrating Windows 10 holographic computers into your operations. This first is the space you’ll use them in. “Space needs vary depending on the use case. For instance, a classroom or training situation might require an empty room with perhaps a few stools, maybe a table or some furniture onto which you can anchor holograms. It also pays to follow high-density Wi-Fi best practices when designing commercial spaces for HoloLens to ensure optimal connectivity.”

You also need to think about how many units you’ll need. “If sharing the HoloLens units, ensure there’s some way to secure, store, clean and charge the devices.”

HoloLens is ready for business

Infrastructure and transport companies are natural early adopters for mixed reality, but Fritjers is expecting to see it become a common tool, and quickly. "In five years I think we will see HoloLens and other devices be, by default, an important way of interacting with models. This will become an important element in the way companies will design projects. Clients will expect in future that if they're going through a design process then they will be able to look into the future, to see and experience what design principles they will use, and walk through their future building."

Although users need to get used to how to navigate the HoloLens system (using voice and the ‘air tap’ gesture that’s the equivalent of a mouse click), Sparked has found that clients have picked it up pretty quickly. "People are used to looking at and interacting with 3D objects," Fritjers said. “The feedback we got is that it was pretty natural and made complex discussions around design options very short, which saved the client a lot of time. The time to market was reduced and so were the risks. They're convinced it's the way forward and they want to use HoloLens with their customers for designing new projects."

"The device has a big wow factor, but the technology is not the goal; it's just an instrument to improve and transform the way people work."

Related Video: Hands-on with HoloLens Development Edition

This story, "Why (and how) to start planning your HoloLens apps" was originally published by CIO.

Computerworld's IT Salary Survey 2017 results
Shop Tech Products at Amazon