Let us face the facts. Even as late as in January 2010, Bing was considered to be the third search engine on the web by query volume at 3.16%, after its competitor Google (85.35%) and Yahoo (6.15%). Nevertheless, Microsoft and Yahoo had already clinched a deal through which Bing would power Yahoo! Search.

As of now, you must be aware of the “Bing It On” challenge with a claim that users prefer Bing over Google 2-to-1, disregarding the fact that Google has momentous market share hovering around 67% (current comScore data) while Bing hasn’t been able to cross over 18%.

Now, let us take a peek at what Microsoft has to say in favor of Bing.

“Since we launched Bing we’ve been hard at work to deliver relevant, useful search results that help you go from searching to doing. Awhile ago, we began to notice an interesting trend in our internal testing – for the first time our testing showed that Bing’s web search results were better than Google’s. We continued testing our results in several different ways as part of our regular work to improve our quality, and along the way an interesting pattern emerged – Bing was regularly beating Google in web search results quality.

However, when another research company was asked to do a statistically significant similar challenge test and our findings were confirmed – people preferred Bing web search results nearly 2:1 over Google in blind comparison tests. Since relevancy of search results is the #1 driver of search engine preference, the time is right for a wake-up call for searchers – better web search results are available at Bing.com”.

But independent opinion in the Bing/Google debacle tells a different story. As a matter of fact, Ian Ayres (on the Freakonomics blog) openly questions the validity of the “Bing It On” challenge, especially how they managed to run it.

Here are the details

First off, is the fact that their statistic states users chose Bing over Google was based on a mere 1,000 users – not a very large pool at all. And they don’t specify how these participants were selected, other than they were 18 and up and from across the U.S. Were these people who are signed up for doing surveys, as there is quite a market for people getting paid for doing surveys online. Or were they selected in some other way?

Nevertheless, it started the ball rolling when Ayres, along with four Yale Law students, decided to do their own survey. But what they found was quite exciting.

Here are the results

When users were selecting the search terms from the ones Bing suggests, the results shown seem to be much more favorable for Bing, leading to speculation that either these results were hand picked or just happen to be pages that previous challenges had shown a marked preference for the Bing results.

In fact, when Bing-suggested search terms were used the two engines statistically tied (47% preferring Bing vs. 48% preferring Google). But when the subjects in the study suggested their own searches or used the web’s most popular searches, a sizable gap appeared: 55-57% preferred Google while only 35-39% preferred Bing. These secondary tests indicate that Microsoft selected suggested search words that it knew were more likely to produce Bing-preferring results.

Bottom line: No matter how heavily Microsoft may beat its own drum in favor of Bing, Yahoo still remains as the topmost search engine till today.