Apple’s HEIF image format choice reinvents photography

The transition isn’t only about image size and quality

Apple, iOS, iPhone, Mac, iPad, photography, HEIF, JPEG, AR, VR

Apple announced plans to adopt the HEIF image format at WWDC. When it did it pointed at the format’s higher-quality and much smaller image sizes, but I think there are big implications beyond this.

A new future for photography

If you’ve read your Harry Potter novels you may recall the picture the young wizard had of his Mum, Dad, and himself. They seemed alive in the image. It seemed like magic. These days it seems like an Apple Live Photo.

A still image that is also a video, Live Photos is a perfect example of the kind of image-based technologies HEIF will enable in future Apple hardware and software products.

At present the way it works depends on several technologies and a series of images, but with HEIF the image file can actually be a real video (and audio, and many other things).

That’s because the format is different than JPEG – while JPEG is a clever way to digitize a still image, HEIF (with its roots in the MPEG-developed HVEC video standard) is a containerization technology, capable of holding lots of different kinds of information inside one data file.

The move to use this container technology will make it much easier to extend what Live Photos and photography in general can achieve. At low image sizes.

What’s inside?

To focus on what I’m trying to say, I’m skipping many technological details here, but do read this excellent article if you want to learn more about HEIF.

You’ll learn that as well as multiple images, an HEIF file can gather image collections, audio information, image thumbnails and also – and this is important:

“Image derivation information such as rotation, overlay and grid view along with the images, so that different image derivations can make use of the same image data set,” and “auxiliary image data such as depth map and alpha channel along with the images.” (Source: Nokia).

Stop for a short moment and think about the implications of that: this means that your photograph can capture depth, alpha channel data, planar information…

Essentially this means that as well as providing better quality (16-bit v 8-bit) images at half the file size of JPEG, HEIF can capture the information you need to place the subject of your image in the space.

Now imagine how this kind of data may become useful within implementations of Apple’s other big WWDC reveal…

Augmented and virtual reality

ARKit is going to be a work in progress for a while, but Apple’s move to adopt a significantly more flexible image format makes it possible to imagine several different ways in which it will become so much easier to bring still images to virtual life.

As well as Live Photos we’ve also seen a couple of other cool illustrations that show how Apple’s photography teams see the future of images:

  • The iPhone 7 captures 10-bit images – it already makes better images than the existing JPEG format can handle.
  • Portrait Mode is a great example of how Apple is trying to develop a photographic system that exploits depth of field and alpha channel data to make sophisticated camera effects available in a mobile device.
  • The Clips app is an excellent illustration of how you can use complex, short, moving images to quickly create viral videos.

How can you exploit it?

Using these examples to guide my thinking, I can already imagine some of the different ways Apple and its developers – including enterprise developers – may be able to use this new image format to support their apps.

  • Think about product catalogs – we already know IKEA is working with ARKit to create solutions that let you virtually see products in your home: How much more effective will this be if you can also scale and plane those product images accurately?
  • Think about photography: Does the gathering of this data not also suggest that you’ll be able to place images of the people and places you most love within virtual environments? Of course, it does and you should be able to see them “as if in the flesh” (adult, or otherwise) with a pair of VR glasses. Think how valuable this might be to people’s relationships with each other, or in terms of future photography.
  • Think about product design: How effective might this be in terms of rapid prototyping and field testing of new products?

Seize the creative opportunity

I’m just a scribbler of words, but I am certain Apple’s move to adopt this powerful new image format mean that we’ll see a deepening connection between the photos we take daily and the introduction of advanced technologies that make such images virtually real.

I think it’s time to stop seeing the move to the new image format as being solely about image quality and file size – those containers can be used to do so much more.

That’s why developers, enterprises, and you and I should begin to think about how we can exploit the huge quantity of data held inside these files. How can we climb aboard this new creative opportunity? How can we create still images that work on their own account, and yet exploit the image data carried inside them to reinvent photography? 

Google+? If you use social media and happen to be a Google+ user, why not join AppleHolic's Kool Aid Corner community and join the conversation as we pursue the spirit of the New Model Apple?

Got a story? Drop me a line via Twitter or in comments below and let me know. I'd like it if you chose to follow me on Twitter so I can let you know when fresh items are published here first on Computerworld.

Computerworld's IT Salary Survey 2017 results
Shop Tech Products at Amazon