Cortana: anticipating a true conversational interface for Windows

To make MR truly immersive, Windows will need to evolve from a Graphical User Interface into a Conversational Interactive Interface.

Microsoft Mixed Reality Headsets
Current Job Listings

[Disclosure: Microsoft is a client of the author]

This last month Microsoft, with a variety of partners, announced a series of low cost VR/MR headsets due to market largely targeting consumers by the 4th quarter.   MR is the new term that refers to Mixed Reality and now effectively encompasses AR, or Augmented Reality, and VR or Virtual Reality. 

These headsets solve many problems plaguing the VR market in particular: initial cost (they cost a fraction of the headsets currently being sold), ease of setup (they have cameras built in rather than the earlier remotely installed satellite cameras) and introduce more hardware competition (which often drives lower prices and more innovative designs). But we still are mostly dealing with interfaces into our new VR world that seem mouse or touch based even though neither really lends itself to VR, MR, or AR. 

They solved this problem in game with the Cortana AR interface they gave their Master Chief character in the game Halo.  While there are still often menus the primary way the helmeted warrior interfaces with his computing resources in conversationally with a female avatar.   

One of the most compelling parts about MR, as a primary tool will be the interactive voice interface and likely force an evolution for Windows. 

Let’s talk about Windows potentially evolving into Cortana.

Digital assistants are the current and next big thing

We are up to our armpits in digital assistants now, mostly based on Amazon Echo.  This idea of interfacing with our technology vocally was first largely championed by Apple Siri but with Echo you really don’t need an iPhone, or any cell phone, and when you need an answer to a question you simply must ask.  No buttons to push, not mice to chase across the desk.  

But it was Microsoft with Cortana in the video game Halo that first seemed to showcase what a digital assistant could be.   Back in May a developer created a 3D holographic Cortana and I’m kind of surprised I haven’t seen this product show up on one of the crowd funding sites of yet.  Imagine your digital, rather than being a piece of hardware, a fully animated woman you could see in 3D without classes.  

But given one of the big problems with consumer MR now is immersion, creating a virtual experience that seems real, the industry must create more realistic controllers and interfaces.  These controllers and interfaces need to replicate better the real world and, in the real world, your tennis racket, gun, or fishing rod isn’t also a mouse.  

These things start to truly get interesting when we back them up with Deep learning systems and allow them to evolve to meet the needs of their users.  

Avatar evolution

As we move away from the idea of a mouse and keyboard interface, and event start to look beyond touch, the idea of a conversational interface gains even more potential once was allow it to learn and evolve.  The nice thing, well one of them anyway, about Deep Learning systems is that they get smarter over time and they can also learn from each other.  This suggests that a future version of Cortana would become more conversational, more understanding of moods and personal events, and eventually evolve to be able to create the perception of real companionship. 

Given how we tend to anthropomorphize things anyway, the idea that we might befriend an avatar that increasingly behaved and acted human shouldn’t be a huge stretch.  Expected as one of the big changes in the coming wave of ever smarter AI based assistants it will herald an era where increasingly machines learn how best to work with us rather than the other way around.  

Wrapping up

With the move to MR I’m anticipating an evolution of Windows from the mouse, keyboard, and more recently, touch interface to one that is more conversational.  Much like when Windows 95 came out and the UI stopped being an optional overlay, Cortana will move from edge of the screen to become the primary interface in that coming virtual world.  She will eventually learn how best to work with us and our relationship with our computers will largely shift from human/tool, to a more collaborative peer like experience as the AI evolves and eventually becomes a far more like a partner and far less like just an annoying tool.  

It should be an interesting future.

This article is published as part of the IDG Contributor Network. Want to Join?

5 collaboration tools that enhance Microsoft Office
Shop Tech Products at Amazon