correlate

Before getting into any of the nuances and complexities of some of technologies we see leveraged today in the Web 2.0 world, it is very interesting when you simply look at ’search’, something that we’ve all been around for years. When you boil it down to the most simple unscientific level, isn’t search relevance simply high correlations of a document found to ‘meeting the users needs’? Thus, couldn’t one argue that the very “best” article/document found from a search query has a correlation of 1.0 with the user’s expectations?

Take a look at a study done by a faculty member at the University of Technology in Sydney comparing the correlation of results by contextualising search engine with conventional engines. The premise of the hypothesis is that “an automated analysis not only of the content of candidate documents, but also the content of related (i.e. either directly or indirectly linked) documents and the structure of the relationships can be used to improve the effectiveness of search engines (Ellis 1996).” As the study points out:

While the results are preliminary, the improved correlation of results in different searches does shed light that contextual searching is a value added way of standard searching. Pretty interesting about the power of context, something much more difficult to achieve off web…

The paper was written in 2000 which makes it interesting that Google is not even mentioned when the likes of Yahoo!, Magellan, AltaVista and Lycos are. Hey, for that matter, why not About.com, we were still considered a search engine back in ‘00 even when we were using Inktomi as our core engine. Back to the point, Google via Page Rank truly pioneered by innovating search and contextualizing it through the power of linking and basing relevancy accordingly. It seems so intuitive now but you really have to give them credit for coming up with it then.