Microsoft and a Yale professor have been engaged in a nasty spat over the company's "Bing It On" blind test which Microsoft claims shows that people prefer Bing to Google. Microsoft clearly beats Yale here. But it's not for reasons you might think.
In Microsoft's Bing It On site you perform searches, and see results from Google and Bing simultaneously. The site masks which search engine the results come from. You then vote on which results are superior, or rate them a tie. It's a clever campaign, modeled in some ways on the "Pepsi Challenge," in which people did blind tastings of Coke and Pepsi side by side. Pepsi, naturally, turned out to be the winner.
In the Bing It On campaign, not surprisingly, Bing was the winner by a nearly two-to-one margin, Microsoft claims. Microsoft has launched a national ad campaign around the results.
Enter Yale law and economics professor Ian Ayers. On the Freakonomics blog, he charges that the Microsoft results aren't believable, and says that in his own testing, using four Yale law students, people preferred Google to Bing by 53 percent to 41 percent, with six percent of votes resulting in ties. Ayers also faults Microsoft's methodology for achieving the results.
Microsoft fired back in its own blog that Ayers' analysis was inaccurate, and that in any case, the two-to-one figures Microsoft used were from controlled studies, rather than from searches done publicly on the Bing It On Web site.
Who to believe? Who cares?
The truth is, Microsoft comes out the winner here, by a long shot. It doesn't matter who you believe in this case, because Microsoft is accomplishing exactly what it set out to do: Get publicity for Bing, and get more people to try it. The spat with the Yale professor gets Microsoft more publicity, and drives people to try the challenge, and therefore try Bing. Advantage: Microsoft.
For what it's worth, I took the Bing It On challenge myself, and did five side-by-side blind challenges. One search was for an unpleasant medical condition, one for cheap flights to Spain, one for "federal budget shutdown" one for Mickey Mantle, and one for Pablo Picasso.
My results were a dead heat -- two for Bing, two for Google, and one a tie. That generally reflects how I feel about the two search engines. They both do equally good jobs. That tie, though, means that Microsoft won. Because the real reason they launched the Bing It On campaign was to get people to use Bing more. I generally use Google. But after taking the test, I'll start using Bing again as well. My guess is that I'm not alone.