Amazon last week introduced a new approach to in-store technology and strategy, with its Amazon Go experiment. On the plus side, it offers a vision of the possible, with shoppers taking a few seconds to check in to a store on their phone and then they grab what they want off the shelves and exit the store.
There's no scanning labels or barcodes and absolutely nothing resembling a checkout lane. Everything is done, in Amazon's phrasing, with "computer vision, sensor fusion and deep learning" and "machine-learning" plus artificial intelligence. OK, but the absence of meaningfully specific technical details has more the feel of smoke and mirrors. (Note: Computerworld asked for an Amazon interview to go deeper into the particulars. Amazon replied and referred us to its website, which we interpreted as "no interview." We get nervous when a retailer offers something vague and then doesn't want to answer any questions about it.)
Yes, there are certainly some competitive issues here, where a retailer doesn't want to teach its rivals how to do this on their own. Still, shoppers have the right to know how closely they are being monitored and the means by which the system concludes that something has been purchased.
Let me try and get specific. It seems clear from a video that Amazon published that the process starts with opening an Amazon app, which will display a barcode or QR code. The shopper displays it to a scanner near the entrance of the store. So far, so good. The shopper is identified to the store and the store is identified to the app. A clean dual handshake.
With video camera and aisle sensors, it's not that hard to determine that an item has been removed from the shelf. The tricky part is determining whether it was purchased. What if a shopper picks the item up and drops it next to the cart, where it falls to the floor? Will an A.I. app watching live video interpret that incorrectly as an item going into the cart? Are there RFID tags on the items and a scanner in the cart? Given that no mention was made of RFID and one mention was made of a sensor, it would seem that Amazon is guessing whether an item went into the cart.
And what if the shopper places it in the cart and later changes her mind, taking the item out of the cart two aisles later and placing it in some random spot? Any grocery manager will tell you that that is far from a rare happening. Cameras may figure out that something was placed on the shelf, but depending on how much of the item the shopper's hand is covering, it may not know what.
That's all OK, though. As Amazon's reference to deep learning and machine learning implies, this is a system that will get good after repeated use. Put pessimistically, this effort will have to fail quite a few times before it gets it right.
The bigger concern, then, is what mechanisms does it have in place to deal with those inevitable fails? Let's not forget the potentially worst-case scenario of new payment technology meets loss prevention (LP) procedures. That was back in 2012, when Apple literally arrested a New Yorker for not properly handling the then-brand-new Apple Store checkout process. The guy had scanned an item for checkout and had forgotten to hit the final button. Instead of asking him to do it again the right way, Apple LP arrested the guy and he spent the night in jail. Amazon, take note.
The video shows shoppers briskly walking out the door, at which point the app displays what it thinks the shopper has just purchased. What is not addressed is what happens if it's wrong? Yes, the shopper can call Amazon customer service and dispute the charge, but what are customer service's marching orders? Do they refund anything disputed until the system works out its kinks? Does customer service have access to the video from the store — video relating to that customer's activity on that day and time and in that store?
If customer service has access to that video and has the tools to zero in on the instant of specific purchases, it could have a very effective tool to resolve disputes. It could even send those video clips back to the shopper for rebuttal. But given that nothing about this was mentioned, it seems unlikely. After all, if Amazon was using video in this way, it would want to advertise that fact, to cut down on the number of true shoplifting attempts.
The less impressive interim technique would simply be to station one employee at the exit. Customers would be asked to look at their app's list of purchases before leaving. If they had any dispute, the store associate could look at their basket/bag right then and there and fix any problems. If true shoplifting is suspected, they could even ask to inspect anything else the customer is holding. That's where disputes can be most easily resolved, assuming the video effort is not an option.
These dispute resolution issues notwithstanding, Amazon's effort is indeed exciting. This addresses some of the biggest problems in-store retail faces. Imagine how much easier holiday shopping would be if every store used this kind of an approach? Envision a Walmart or Target trip without any checkout at all? Heck, let's envision this for the warehouse chains (Costco, Sam's Club, BJ's) where the weekend checkout time can easily top the rest of the shopping trip time.
The benefit here isn't limited to getting rid of checkout lanes. Video of everything — searchable for every specific purchase — is a major LP advance. That video also is a huge foot forward for CRM and marketing. It not only notes every purchase made, but every purchase considered but rejected. It stores how long a purchase was considered and can note what was the last thing examined — such as an ingredients list — before the purchase was abandoned.
Amazon may have a few bugs to work out, but this Amazon Go trial has some extremely encouraging potential.