Life on The Long Tail – Why a Power Rule Curve for Web Pages?

Links to web sites follow a power rule curve.  Using Technorati ranks for links to blogs, #1 is geocities.jp (Japanese) which has 119,246 blogs linking to it, #2 is community.live.journal with 75, 351 logging in (power rule predicts 60,000 or half of #1), #3 is videocodezone.com with 26,549 (prediction 40,000) and #4 is Engadget with 25,627 (prediction 30,000).  Average out the variances and you get a pretty good power curve even though the top sites are actually aggregates.

Who enforces this rule?  Why do links to web sites follow a power rule curve.  Turns out that Technorati, and Google, and you are all responsible.

Just by listing blogs in order of inbound links, Technorati helps keep the power rule in force.

Part of Google’s not-so-secret sauce in its search algorithm is to give more weight to those sites which are frequently linked to by other sites.  But not all links are equal.  Links from authoritative sites count more than links from non-authoritative sites.  But what makes a site authoritative?  More links to it – especially from other authoritative sites.

Any time you Google you get back a list with high-ranked sites which contain your search terms.  You look at these first because Google returns them first.  If you are blogging, you are more likely to cite these sites than the ones on page 10 of Google results because you’ll probably never get to the tenth page.  You just helped the rich get richer and helped enforce the power law.

You may even have a self-serving motivation of your own.  You know that powerful sites you link to may possibly link back to you – better chance than if you don’t link to them.  By the way, writers of scientific papers take pains to cite papers by the most eminent members of their discipline for exactly the same reason – and because these people may be in a position to give them tenure.  The initial Google algorithms were consciously modeled after the academic system of citations (read John Battelle’s The Search and search blog to learn all about that).

Moreover, some search engines and blog rating services give more weight to links TO authoritative sites than links to less authoritative sites.  People do the same.  If I quote John Battelle on search or Chris Anderson on the long tail, their quotes have more weight than if they came from people you never heard of.

But don’t blame yourself or Technorati or Google too seriously.  Links to web pages and views of web pages followed a power curve before there was a Google or a Technorati and before either you or I started blogging.

We are the mechanism for implementing something that nature seems to like – the power law curve.  It applies to the distribution of wealth in free societies, to sales of many products even before there was an Internet, and to the frequency of the use of individual words in languages (in fact where it was first noticed).

Good news on the web, at least, is that positions in the power curve are hardly permanent.  There is plenty of turnover and are many position changes in the top 100 blogger list from year to year.  Star status is vulnerable.  New stars are born.  But, the bigger the blogosphere gets or the web gets or the longer any other tail gets, the harder it gets to move off the tail to the head.  Clay Shirky, expert on web power curves, said: “It's not impossible to launch a good new blog and become widely read, but it's harder than it was last year, and it will be harder still next year.” By the way, he wrote that in 2003 when blogs were young.

In the same post he also wrote: “Inequality occurs in large and unconstrained social systems for the same reasons stop-and-go traffic occurs on busy roads, not because it is anyone's goal, but because it is a reliable property that emerges from the normal functioning of the system. The relatively egalitarian distribution of readers in the early years had nothing to do with the nature of weblogs or webloggers. There just weren't enough blogs to have really unequal distributions. Now there are.”

This series about life on the long tail starts here and continues here.