Is Amazon trusting tech too much?

Amazon envisions a retail floor with no checkout, run by an array of cameras and scales that will identify shoppers and the products they grab.

Amazon tape

When Amazon recently filed a patent application to make retail payments absolutely transparent to shoppers, it displayed the same shortcoming that has undone many an IT project before now: having too much faith in bits and bytes doing what they are supposed to do.

The idea is to use video cameras — and a host of other technology, including RFID and a wide range of sensors — to identify shoppers and the products they grab and to then charge those customers when they walk out the door, all without asking those shoppers to even slow down. The intent of the filing is to have an entirely automated system, one that doesn't even require store personnel to monitor systems from the backroom or a centralized call center. That means using technology to determine what human eyes normally would.

The filing spoke of using pressure sensors to "detect a shape or dimension of the bottom of the item as it sits on the pressure sensor. The shape of the bottom of the item may be compared with stored dimension information for each item on the item identifier list."

The patent paperwork envisioned using microphones to "record sounds made by the user and the computing resources may process those sounds to determine a location of the user. For example, based on knowledge of the locations of the microphones within the materials handling facility, a time offset between audio signals received by each microphone can be computed to determine a location of the user."

Here's where things get overly trusting. The system wants to use a shopper's purchase history to help give clues to the identity of items that are placed in a cart. The application talks about using "other information about the user (e.g., past purchase history, currently picked items) to assist in identifying the item. For example, if the inventory management system cannot determine if the picked item is a bottle of ketchup or a bottle of mustard, the inventory management system may consider past purchase history and/or what items the user has already picked from other inventory locations. For example, if the user historically has only picked/purchased ketchup, that information may be used to confirm that the user has likely picked ketchup from the inventory location."

The ketchup example raises accuracy concerns. What if shoppers choose to deviate from their purchase patterns that day and opt for a similarly shaped but much more expensive product? A key retail concern will be a means to positively identify products and shoppers. That should be addressed by RFID tags, but a version without RFID would still need backup authentication systems.

Today, that is done by the sales associate at checkout, who can verify identity and make sure that what scanned through as a bag of grapes is not in fact an HDTV — which is the oldest trick of the shoplifter who switches barcodes.

Being overly trusting was one flaw. Being politically naive was another. Consider, for example, how the patent application described a video analysis system intended to guess if the shopper has just picked up an item.

"Image analysis may be performed on the first image to determine a skin tone color of the user's hand and pixels including that color, or a range of colors similar to the identified skin tone color, may be identified to represent the user's hand. Utilizing the skin tone colors, the images of the user's hand obtained after the user's hand is removed from the inventory location may be processed to again identify the user's hand," the filing said. "Finally, a comparison of the segments of the images representing the user's hand and an area surrounding the user's hand may be compared to determine a change between the images to identify whether an item has been picked or placed into the inventory location."

It's not clear why the shopper's skin color would play a role here. Yes, it represents the user's hand, but the skin tone would likely be similar on the forearm and other places. That wording seems guaranteed to bring Amazon some grief. Then again, this is from the same legal group at Amazon that brought us — no kidding — a way to guess a shopper's religion based on the giftwrap they choose.

Here’s another phrasing, this one likely to ingratiate worker leaders. In retail, store employees are almost always referred to as store associates. Amazon, however, doesn't seem to think an associate has to be alive.

"If it is determined that the user did not confirm an identity of the item placed into the inventory location, an associate may be dispatched to the inventory location to retrieve and/or identify the placed item," the filing said. "In some implementations, the associate may be a human. In other implementations, the associate may be an automated drive unit and/or a robot configured to manage and/or handle inventory."

In some implementations, the associate may be a human? They couldn't have instead said "in some implementations, an associate could perform that task, but it might also be done by a machine"? Sounds like Amazon's Legal department could use a bit more sensitivity training.

Computerworld's IT Salary Survey 2017 results
Shop Tech Products at Amazon