If you watch television at all, you’ve probably seen the commercial for Microsoft’s Bing it On campaign (if you don’t watch television, you should check it out, there’s some great stuff on there, I personally recommend Breaking Bad.) In case you haven’t seen it, Bing it On is essentially a blind comparison between Bing and Google, a la Pepsi and Coke back in the day. A search is performed, two result windows pop up, and you select the one you think delivered better results. In the commercial, of course, Bing is the clear winner.
But are the commercials telling the truth?
(No.)
Ian Ayres (of the Freakonomics blog) had some serious doubts about the Bing it On Challenge, and decided to run some scientific tests. They immediately found several flaws with Bing’s process, such as:
- Small sample size (only 1,000 users)
- No indication as to selection method
- No personalized results that Google traditionally delivers
- No geographic results that Google traditionally delivers
These facts alone are more or less enough to invalidate Bing’s claim, but there is more. It turns out, when Bing suggested the search term for the test, the two search engines were essentially tied, with either one taking or losing the lead interchangeably. However, when the user was allowed to decide on their own term and run the test, Google had nearly a 60% preference rate. That pretty much proves that Bing was hand-picking search terms it thought it could compete especially well for. The sad part of that? Even when it was hand picking terms, it was still only tied at best.
The paper by Ayres is a lot longer, and has been removed from the internet.
Microsoft has responded, saying that Ayres study is also not scientific and that his results shouldn’t count either. Who is right? Personally, I think Microsoft is being misleading with their numbers, but I encourage you to check out Bing it On, look through Ayres’ paper, and come to your own conclusion.
Either way, Google is still the search engine to focus on for SEO.