Skip the navigation

Evan Schuman: Killer robots? What could go wrong? Oh, yeah ...

The UN wants to talk about killer robots as 'conventional weapons.' Someone needs to learn the IT facts of life: If something can go wrong, it will.

By Evan Schuman
May 14, 2014 09:49 AM ET

Computerworld - Has anyone at the United Nations ever used Windows? This question occurred to me after reading an article with the headline "'Killer robots' to be debated at UN."

This was the kind of news item that makes you double-check the URL to make sure it's not from a satirical site like The Onion. But no; it was from a real news organization, the BBC. According to the report, "Killer robots will be debated during an informal meeting of experts at the United Nations in Geneva" later this year.

The report defines a killer robot as "a fully autonomous weapon that can select and engage targets without any human intervention." The BBC adds that "the meeting will be held during the UN Convention on Certain Conventional Weapons." It's probably best to not ask when the UN decided that killer robots are now "conventional weapons."

I find this frightening -- and Windows has a lot to do with that. Well, the real reason is that I'm familiar with the IT facts of life. Websites operate perfectly fine for months and months and then, without anyone touching one line of HTML code, they suddenly glitch. Windows desktop units, years after blue screens of death were supposed to be extinct, freeze without any apparent cause. Mobile apps are developed and rolled out with egregious security holes.

And now we're talking about autonomous killing machines? Talk about your killer mobile apps! These could come with missiles and automatic weapons. The idea is that these devices would sharply reduce casualties, at least from the side that launched them. The problem is doing that while making the fatal IT assumption that computer programs never fail, fail-safes are always reliable and nothing will be overlooked. Sound like any IT operations you've ever seen on Earth?

Here are some more annoying bits of reality: cyberthieves, hostile governments and terrorists. Bad guys have been known to break into sites and then take over (scripts are far easier to trick than even government workers), with freedom to access -- or destroy -- unlimited data.

The technology hurdle that has to be cleared before we, as a society, even consider killer robots is this: strict quintuple-verified authentication. The robot's program must have a foolproof means -- with as many layers of backups and fail-safes as possible -- of verifying any orders it is given. Autonomy will have to be limited at least to the extent that the robots will respond to self-destruct or shutdown commands. But if the robots are truly autonomous -- well, just stream I, Robot or The Matrix if you're unfamiliar with the sorts of nightmare scenarios sci-fi has already shown us. And when it comes to killer robots, I think a nightmare is inevitable.



Our Commenting Policies
Consumerization of IT: Be in the know
consumer tech

Our new weekly Consumerization of IT newsletter covers a wide range of trends including BYOD, smartphones, tablets, MDM, cloud, social and what it all means for IT. Subscribe now and stay up to date!