Last week, AV-TEST, an independent IT security institute based in Magdeburg Germany ran malware tests on four top search engines, to see which engines provided more malware in search results.

The end results are on the graph above – while Yandex and Bing Search did have more websites tested, they also had a lot more malware per search. Both Bing and Yandex have responded to claims their search engines are less secure than Google’s own, assuring users they are safe.

Microsoft Denies Test Results


David Felstead, Senior Development Lead at Bing, responded through Bing’s Blog to the test, stating the results are false and AV-TEST ran the tests in an indecisive way, through API results.

The first point to make is AV-TEST doesn’t do a traditional test, that would be going onto then testing a load of searches. They simply run Bing’s API and test millions of searches and then check for malware.

Bing’s API does not display warnings when the users click on a website and bypass the defenses Bing has placed. This means for the average user, the malware will be blocked behind warnings.

This still doesn’t take away the problem Bing has more malware on its search results – these are top searches and the websites contain some malware that could infect the system if users are not careful.

Felstead then goes on the offensive against Google, its main competitor. Vacation Hotline, a well-known malware ridden site, is still open to view on Google and Yandex, yet on Bing, it is plugged behind warnings.

Overviewing the amount of malware on a system is not good enough, according to Felstead, because many hacked websites have malware but Bing keeps them open because they are legitimate websites, which could be fixed.

Felstead also goes on to talk about how malware is a complex problem both Bing and Google are working tirelessly to fix – Bing does a good job and you will only ever see a warning appear once every 1 in 10,000 searches.

Overall, the message is while Bing may have more malware, it takes care of it with security and defense better than competitors. This allows users to have a safe experience and not one filled with malware warnings.

Yandex Also Denies Test Results

Yandex, the Russian search engine, has also hit back at claims their search engine is riddled with malware compared to competitors. Instead of bringing up good and bad points about the company, they ask some valid questions about the testing and results:

  • How did they define what is malware and what is not?
  • Did they take into account the fact that Yandex does not remove potentially malicious websites from its search results page, but rather notifies a user about potentially dangerous sites with a special mark in front of the link?
  • Why is the sample volume of websites so different for some of the search engines tested?
  • What does “malware found” mean? Is it what a search engine found and marked or is it what a researcher found unmarked by a search engine?
  • What was the way for them to collect samples – is it XML or just regular parser?

This is the same sort of idea Bing has, not closing websites but monitoring and notifying users. The idea of malware found is pretty important too, AV-TEST has yet to identify what “malware found” means.

What does this all mean?

The problem with tests of this scale is the defining factors and analyzation. So far, AV-TEST haS not discussed a lot of features they have within testing to make sure results are valid.

Malware is not as big a problem as it was 5 or 10 years ago, in fact, it is becoming submerged on Android as the web becomes a cleaner area. Google, Bing, and Yandex all do an excellent job to prevent the problems which could emerge from malware.

For the general user, these tests will be inconclusive to the best search engine, because the raw amount of websites tested, the testing they went under and the defenses each search engine has blocked most of this malware.