Well Connected

Image courtesy

In a related post to the one below, there was an interesting feature in this weeks New Scientist about a scientist who is using the characteristics of networks and linking in fascinating way. Peter Gloor at MIT (he of Swarm Creativity) has developed software, called Condor, which uses a property of networks on the web he has termed "betweenness" to make predictions.

It begins with taking an ordinary search term, plugging it into Google, and then taking the URLs of the top 10 returns, plugging them back into the search engine prefaced with the term "link". Google then returns the sites that link to the ten orginal sites, and this process is then repeated with the new sites. The software then maps the links between all the sites and works out the shortest way to get from one to the other via the links they contain. Sites score higher betweenness scores the more often they link to other members, and an overall score for the original search term is calculated by averaging the betweenness scores of all the sites.

Using this score alone can be a good indication of popularity (the example given involved Gloor entering a range of film titles for a whole year into Condor. Of the ten which acheived the highest betweenness scores, five won oscars, four were nominated and only one got nothing). But improvements are now being made to Condor to allow it to search blogs and chat forums in isolation so that these scores can be weighted alongside scores from the web to acheive a more accurate result. Surprisingly, the magic combination is believed to be 15% from the web, 5% from blogs but 80% from discussion forums - a result he puts down to the belief that forums tend to be populated by people who are most interested and have the most to say. Either way, he's acheived some impressive predictive results using Condor including predicting 3 days ahead to an accuracy of 80% whether companies stock prices would go up or down, and it's also been used to predict the result of an Italian political party's internal election a month in advance to a degree of accuracy equivalent to exit polls.

It feels to me like we are just scratching the surface of what may be the true potential for communications in developing a better understanding of network properties and effects. In the meantime, perhaps they should be using it for Obama/Clinton?