Subscribe to the RSS feed (english)

Relevance and reputation, votes and grades

Through Fred Cavazza (in French), I discover this interesting article of Wired at the very moment when I was thinking to continue my series on blog relevance.

I have already studied two ways to estimate the relevance of a content:

Here is the third (and most commonly used) category: the grades.

These systems require their members to grade (between 0 and 10, for example), contributions or directly other members. In the same family of tools, you can find the voting systems: a vote corresponds to a binary grade (0-1).

Then, the system computes an average, to estimate the value of that content or that member.
The most commonly quoted grading systems are Digg and Ebay.

In the first case, we vote for an article, in the second, for a seller or buyer.

These systems are heavily criticized because they are easy to hack. Wired's article describes several methods to cheat on Digg or on Ebay.
It will be objected that fraudsters can be detected, and their accounts deleted. But when they get caught, it is often too late: buyers have purchased imaginary products or bad articles have received undeserved audience.

What is the problem of these systems?
The grades and the votes are too easy to give. Robots can vote automatically. So we can create robots that filters robots bypass. And if robots do not vote, friends can. In fact, only one click costs them nothing, or at most a few seconds.

If you want to build a system that cannot be hacked, either you make it unbreakable, or you make it so that the cost of breaking it would get higher than the expected return.

Let me explain.
Buying votes is equivallent to paying a few seconds of "work" of a friend or corruptible member. If, instead, we must not spend a few seconds, but several minutes or even several hours, then the cost of corruptible member soars.

A unbreakable system could be a system for which one must devote a lot of time. Because, as I have said (in french), time is the scarce resource of the Internet.
Scarse, and therefore expensive.

Second problem of such grading/voting systems, their low representativity. Only a small (1% max) portion of people vote or grade an article. So, 99% of readers are not taken into account. Are the 1% representative of the whole readership? Hard to know, but very likely no.

So, a more reliable grading/voting must take into account more people and require more time to cheat on. And the time each reader spends reading the article, is a kind of grade, isn't it?