Miraculous Facebook A.I. for blind users paints a word picture

“Image may contain: billionaire, grinning. Image taken several years ago, before Facebook became a terrifying, soul-sucking hydra of monumental proportions. Stop… Stop will you?.. Daisy, Daisy, give me your answer do…

Facebook visual impairment Mark Zuckerberg
Credit: Facebook

Facebook will now assist blind users, by recognizing and describing what's in the images their friends share. Facebook calls it Automatic Alternative Text (AAT), because the artificial intelligence system fills in missing 'alt' text, which can then be read aloud by the screen readers of those with a visual impairment.

[Developing story: Updated 6:06 am and 10:33 am PT with more comment]

In IT Blogwatch, bloggers can see the future. Not to mention: Take a stress pill and think things over

curated these bloggy bits for your entertainment. 


What’s the craic? Megan Rose Dickey waxes metaphorical—Facebook’s tool to help the blind “see” images:

Facebook has launched a tool...for blind and visually impaired people to “see” images on the site. [It] uses object recognition technology to generate descriptions of photos.

Now, someone could hear “image may contain three people, smiling, outdoors.” [It's] based on a neural network with billions of parameters...trained with millions of examples.

AAT is currently available for iOS screen readers set to English. ... Facebook will soon add the functionality to other platforms and languages.


It's about preventing exclusions. At least, that's according to Nic Healey—Facebook is getting better for the blind thanks to clever AI:

Studies have shown that blind people often feel excluded...on social media. ... A new feature from Facebook hopes to fix that.

Back in the early days of the internet...alt text...was an important part of putting your webpage together. [It] is what shows up when images broke or failed to load. [And it] means that blind or visually impaired users...get short descriptions of the images people are posting...via screen readers.

The company has been working on automatic alt text for around ten months. ... It follows Twitter recently allowing users to manually add alt text to their pictures. [It] is only available...in the US, UK, Canada, Australia, and New Zealand.


Let's hear from some Facebook engineers and scientists. Darío García García, Manohar Paluri, and Shaomei Wu let us peek Under the hood:

While this technology is still nascent, tapping its current capabilities to describe photos is a huge step. ... Facebook's mission is to make the world more open and connected. [So] we want everyone to have equal access. [But] more than 39 million people are blind, and more than 246 million have a severe visual impairment.

It is quite challenging to teach a machine to provide...useful information...while acknowledging the social context. [It] is a formidable task given the scale and diversity of the visual material.

The engine is a deep convolutional neural network with millions of learnable parameters. ... We carefully selected a set of...concepts based on their prominence in photos [and] the accuracy of the...engine:..such as people's appearance...nature...transportation...sports...and food. ... Our goal is to keep increasing the vocabulary...and making a more inclusive Facebook experience.


How do you switch it on? Queenie Wong rules, dude: [You're fired -Ed.]

To access the tool, people turn on the...VoiceOver feature. ... Then open the Facebook app.

Facebook...isn’t the only tech firm that is trying to [be] more accessible. ... Smartphones and apps from the iPhone’s VoiceOver feature to...Android’s TalkBack are allowing the blind to become more independent.


So here we go, careering towards a future of HAL 9000 and the rise of the Machines. Steven Loeb thinks not—Facebook uses AI:

A feature like this should remind us of the power that artificial intelligence has to...better the lives of people. ... I have recently thinking about blind people and their role in our ever changing society. We have all this great new technology...but is anyone doing anything to help those who can't see experience it as well?

There's a lot of fear and demagoguery that goes on with new technology. But we should also remember the good it can do.


Update 1: But could it go horribly wrong? Casey Newton gravitates towards the dark side-Facebook begins using artificial intelligence:

Similar technology powers keyword searches in Google Photos and Flickr. ... Last year, Google was forced to apologize after Photos tagged two black people as "gorillas."

It’s a cliché for tech companies to describe a project as "just the beginning," but in this case it feels particularly true. ... The team is already pushing hard on...recognizing objects in videos...and something it calls "visual Q&A." ... You might ask who is in a photo...and it would tell you the names of the Facebook friends who appear.

It’s difficult to imagine Facebook without photos. Yet for millions of...visually impaired people, that’s...reality.


Update 2: Get off my lawn. Or so Andrea Peterson seems to say-How Facebook is helping the blind ‘see’:

The early days of the [web] were actually easier for blind users. ... If a picture shows a couple overlooking the ocean while wearing sunglasses...the new alt text will look something like this: “This image may contain: two people, smiling, sunglasses, sky, outdoor, water.”

Most major tech companies have dedicated accessibility teams. ... Many tech companies, including Facebook, along with several universities are...focused on better preparing students to create inclusive technology.

Getting to this point has been tough. ... The National Federation of the Blind...has brought legal action against tech companies in the hopes of making their services more accessible. ... Those legal actions are generally brought under the Americans With Disability Act. But the 1990 law was put in place before the [web]. The Justice Department has delayed guidelines that will cover...online services...until 2018.

And Finally…

Stop. Stop, will you? Will you stop?


You have been reading IT Blogwatch by , who curates the best bloggy bits, finest forums, and weirdest websites… so you don’t have to. Catch the key commentary from around the Web every morning. Hatemail may be directed to @RiCHi or itbw@richi.uk.
Opinions expressed may not represent those of Computerworld. Ask your doctor before reading. Your mileage may vary. E&OE.

To express your thoughts on Computerworld content, visit Computerworld's Facebook page, LinkedIn page and Twitter stream.
From CIO: 8 Free Online Courses to Grow Your Tech Skills
Shop Tech Products at Amazon
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.