Monday 13 June 2011

Information Police

Having watched the Eli Pariser talk Beware Online “Filter Bubbles” on TED I was taken by the huge impact that filtering can have on the information that is available to us and especially when we map this back to a business context. The balance between the speed of access to information that you need and the context that the information is being returned from.



The context
The amount of information in the world is vast, by 2020 it is predicted that there will be 35 ZB of data according to an IDC White Paper. This is an incomprehensible amount of data. If that was all mp3 files then it would be the equivalent of 190 billion centuries of uninterrupted music; if it were DVDs, that is 2 billion centuries of uninterrupted movie viewing. That is if we continue at today’s pace, which is unlikely, we could be looking at yottabytes of data which is 10 to the power of 24 - staggering numbers. 

Historically information has always needed to be classified in order to make it accessible and retrievable. Robert Cawdrey wrote the first English dictionary in 1604, the first telephone directory was in 1878 and contained a single page, libraries are thought to have appeared in Greece in around the 5th Century BC and maps date from around 16,500 BC - these are examples of systems that can be used to find information in a timely manner and have been utilised for years. 

Today we have a different challenge, volume - It is estimated that there are 2 trillion webpages on the Internet - that is equivalent to 300 per person on this planet. That is a long book. With tagging, taxonomies and other categorisation we begin to be able to find information more easily then trawling through indexes of websites and information. This is where we got to in the late 1990’s. Fast forward to now: with the social web 2.0 explosion there is a myriad of ways to search information and embedded intelligence that can be automated to give us information based on our behavioural patterns - such as where we are, our browser and historic searches etc.

Filter Bubbles
This takes us to Eli’s presentation and a couple of interesting statements, the first is from Mark Zukkerberg:
“A squirrel dying in front of your house may be more relevant to your interests right now than  people dying in Africa.”
Whilst on first reading this sounds in poor taste, I interpret the message as being that everyone cares about important things at the global level but they are more interested in the things which directly relate to them or have a more immediate impact or effect. This as a concept is really important and I think of this as spheres of information and different levels of weighting depending on what your objectives are when you are searching. 

Search engines cannot read your mind or even really predict what you are trying to do accurately so they work based on past behaviours and various signals with complex algorithms to work out what is and is not relevant. This is flawed because it does not have the benefit of understanding your emotions or your objectives at that snapshot in time - search engines cannot tell exactly what type of detail you are looking for. 

Eli’s example of Egypt is a powerful one and also raises lots of moral questions, how much should be filtered based on our past trends of behaviour against what is available? Getting information quickly is the biggest benefit as more often than not this approach will give you what you are looking for very quickly and effectively. But this doesn’t always work as per Eli’s point. Dependent on my objective my personal sphere of interests may be more important than a global sphere. On this, I think that we have several spheres of information in our personal and business domains which map roughly to the following:

Personal 
People who I know well or are in close proximity to my location. 
Subjects that I am very interested in or follow avidly. 
Community
People or topics that are further removed but are familiar to me and frequently accessed. 
Regional 
Things that I am aware of but do not have a major impact on me.
I may be aware of the people and subjects but not in detail.
Global
I have a high level understanding and whilst it might be important it does not directly effect me.

Based on what I am doing I may want to focus more on different spheres or bands of information and here we have the time tradeoff. I recall a few years ago a search engine which allowed you to use slider bars to focus on different areas and essentially add a priority to information. When the search engines or filtering tools are doing this it would make a lot of sense to show this so the user can clearly see what information is in scope and what has been filtered out - a Venn diagram of results would be relatively simple for most advanced users with the context focus being tweaked as the user wants to see more or less of certain spheres. Eli uses the example of dessert and vegetables - at times you want dessert - find information about football, friends birthdays or low brow comedies, other times you want to find out about international affairs or the works of Dostoevsky. I prefer to see these as both valuable sets - whether it is dessert or vegetable information you still want to see it. 

In the business context I come across this all the time - how can you enable an employee to find the information that they are looking for? How can you minimise the steps or hops required to surface the information that they need? The right information, in time and actionable. I do not believe that there is enough done to utilise the types of algorithms used in public Search engines in the business domain and obviously the technology is improving in the business context, I am impressed with what I have seen of Fast, which are limited by the security and other constraints. How often do you have to scour through pages of information to get the information that you need? Wouldn’t it be easier if you could remove the types of pages which always come up and that you have to sift through? So in the business context I think there is a case to be made to have more filter bubbles. 

Regardless of personal or business context, regarding filter bubbles I do agree with Eli that the algorithms and filtering approach should be clearly shown to users so that we know what we don’t know. 

For more info you can reach us here: Technical@5point9.com

Technorati Tags: Filter Bubble, Search, Ted Talks, Eli Pariser, Information Management 5point9

No comments:

Post a Comment