Many of us have known for a while that the search engines, and in particular Google, customise results. What you see on your screen will not necessarily be what someone else sees on theirs even with what appears to be an identical search. Location, browser, search history, your browsing behaviour and your social networks are just some of the factors that are used by Google to personalise your results. I recall sitting next to Marydee Ojala at a business information conference in the Czech Republic in 2008 and under discussion was a search visualisation tool. We both ran the same search – Czech coal production, I think it was – and from the maps that were generated on our screens it was obvious that we had completely different sets of results. Marydee had what we decided was an “unadulterated” set with pages that did include the keywords but were not exactly on topic, for example a report on a local football match sponsored by Czech Coal. In contrast, my results were mostly Czech coal production statistics and news on the energy sector in the region. I regularly research the European energy sector and the search engine underlying the visualisation tool used my search history to adjust the results accordingly.
Fast forward to a couple of weeks ago and a request appeared on one of my discussion lists from Mary Ellen Bates for people to run a search on Israel in Google News and to send her a screen shot of what we saw. 37 people responded within 6 hours and the results are very interesting indeed. Full details are on Mary Ellen’s blog (Is Google really filtering my news? http://www.librarianoffortune.com/librarian_of_fortune/2011/09/is-google-really-filtering-my-news.html) but here are a few highlights:
- One story appeared in more than 90% of the search results, another appeared in 70% …. Of the remaining 14 stories, none were seen by more than 30% of the searchers, and most were seen by less than 15% of the searchers
- More than a quarter of the stories showed up in only one searcher’s search results
- Almost one in five searchers saw a story that no one else saw
- Only 12% of searchers saw the same three stories in the same order
The 6 hour spread of the responses may account for some of the differences. Google is constantly picking up stories and changing the grouping and ranking of the articles. Nonetheless, the results still show that you need to use your advanced search skills and look at more than the top headlines for the full picture. To quote Mary Ellen: “Bottom line: Holy moley, Google does filter the news. You really need to go beyond the first few search results if you want to get a relatively well-rounded view of the news.”
Very interesting article – thanks for posting it.
I’d be particularly interested to know how much this is affected by things like 1) Having a Google account which you are logged into (or not) at the time of the search 2) Maintaining good discipline when it comes to things like clearing cache, cookies and browser history.
Has it been possible to delve into this deep enough to answer these questions? Or is that research still to be done?
Hi Lisa,
Mary Ellen just asked us to run a search in Google News on Israel. I was logged out of my Google account and as she is based in the US I used news.google.con for my test. I did not clear my cache, cookies and I think my browser history was off. I sent those details to Mary Ellen along with the screen shot plus details of browser and version number, and operating system in case they might prove useful but I’m not sure if everyone else did. Mary Ellen just wanted everyone to do the search and screenshot as soon as they received the email without clearing ‘stuff’ to see if the ‘filter bubble’ really did exist and was significant.
I don’t think it is possible to determine how much each element contributes to personalisation, we just know from experience that it happens. Even when I have cleared everything and used Incognito mode in Chrome I still get different results if I run the search several times over an hour because Google is probably sending my query to different servers each time.
But to answer the first part of your question: in general, yes I have found that clearing cache, cookies, web history makes a difference and logging out of your Google account makes even more of a difference especially if Google knows about your social networks.
Thanks for your informative reply – a very interesting experiment and a topic I’ll be interested to learn more about in future.
Thanks for this interesting post. It’s good that you were able to test in this way. In case other readers are interested – Google Help itself does give users some simple instructions for turning off search engine personalisation, for both signed in and signed out searches, here: http://www.google.com/support/websearch/bin/answer.py?hl=en&answer=54048. I haven’t tested how well it works! And of course it wouldn’t have any bearing on the issue of different results over time.
a good thing to check out on this topic is a book called “filter bubble” by Eli Pariser. He talks about the extent of which information is being filtered, mentions many sites which are filtering and gives his reasoning as to why they are filtering. Many sites beyond google is filtering results and most are doing it completely with out the knowledge of the end user.