How is the forum rating system doing?

[Richard]

Currently:
531 members have received at least 1 vote, hence eligible to vote for others.
39 members have received 25 or more votes , hence able to give -ve votes as well.


In the last week:
There were 208 votes in the last week placed by 66 members, received by 85 authors

168 votes were +ve placed by 62 members, received by 79 authors.
40 votes were -ve placed by 9 members, received by 23 authors.

The most +ve received by one person were 16 votes from 9 members - also that person received zero -ve votes
The most -ve received by one person were 5 votes from 4 members - also that person received 6 +ve votes from 5 people

The person who placed the most +ve (20) also placed the most -ve (12)votes
Similarly the next most +ve voter (17) was also second most -ve voter (8)
Curiously the next most +ve voter (13) place no -ve votes

With regard to block voting, I’ve looked at how many members cast multiple votes for another person.

Of the 168 positive votes:
1 vote for a member – 115 occasions by 62 different members
2 votes for a member – 18 occasions (ie 36 votes in total) for 16 different members by 12 different members
3 for a member – 3 occasions (ie 9 votes) for 2 different members by 3 different members
4 for a member – 2 occasions (ie 8 votes) for 2 different members by 2 different members
Checksum = 168 total

Of the 40 negative votes:
1 vote for a member – 34 occasions
2 votes for a member – 3 occasions (ie 6 votes in total) for 3 different members by 2 different members
Checksum = 40 total

Of the 40 negative votes, there were 3 occasions when a person recieved 2 votes for the another person. All the others were single.

[Nando]

And how many votes have you and your moderating crew personally neutralized due to (logical) complaints on why there´s even a downvote on it in the first place?

That´s the fascinating number imo.

[Richard]

Okay, I'll state my position…

I’d agree with all your comments. We do need more people voting because then aggregate effects would iron out idiosyncrasies. That brings us onto the most common complaint when members cry foul for an unusual negative vote. We’re all human and in nearly all cases when I enquire about the negative vote the voter has been reasonable from their perspective.

That’s the crux of it, we all have different perspectives. Some people voted for *insert your of president/prime minister* while other voted against. If people can get something so fundamentally wrong as thinking *he/she* is an *idiot/inspiration* then no wonder we see oddities on here.

What I have seen is that people who write prose and explain their ideas get more votes. It’s as simple as that, most people like engaging in constructive conversation. One liners can be easily misinterpreted, sometimes they read as the opposite of their intent, other times they just seem cheeky and flippant.

Posts with videos, news reports and links to other sites but no comment sometimes get a -ve vote because the voter says that posting PR doesn’t add any value. It’s not engaging with the conversation.

As for Nando’s Pirelli video, he posted it without any comment. One person thought it was another Pirelli corporate video without any comment or addition to the conversation. Another member thought the Pirelli video was worthwhile so they gave a +ve vote. As a result Nando’s post has ended up neutral. On balance, that post was probably about right as a neutral rating and that’s where it ended up. It’s all about the aggregate result.

Personally, I’d advocate using stars to give a broad brush overview and take the heat out of arguing over individual votes. That would enable us to convey that one thread/member is generally more useful than another thread/member. IMHO that’s the only context for this sort of system. Just like any crowd sourced system, trying to forensically dissect every vote is pointless due the huge variety of personal opinion.

That brings me onto those ants, they’re a great model for human behaviour. We all think we’re individuals and special. We all think our erudite thoughts are important. However, when it comes to group behaviour we are as predictable as ants. My firm has a couple of doctorate folk who use that sort of modelling to predict decision making, they’re unnervingly accurate. Those models tell us that we need more people to use their votes so the outliers become statistically less significant. They also tell us that decision making on an individual level is erratic even though the overall outcome makes sense.

[Richard]

And how many votes have you and your moderating crew personally neutralized due to (logical) complaints on why there´s even a downvote on it in the first place?

That´s the fascinating number imo.

In the week of the sample data used in my statistics above … *drum roll* ….. none. There was one complaint that resolved itself when other members gave +ve votes.

It’s a tricky statistic to calculate because we’d have to manually compare the moderating log against voting records. However my impression is that we only get one complaint a week about the voting

Ps – It’s Tomba’s moderating crew

[Nando]

In the week of the sample data used in my statistics above …

What week was that?

[Richard]

Noon GMT on Thurs 21st March to noon on 28th March - This was deliberate to make sure it started with a race weekend and the resultant activity.

[Pup]

That brings me onto those ants, they’re a great model for human behaviour. We all think we’re individuals and special. We all think our erudite thoughts are important. However, when it comes to group behaviour we are as predictable as ants.

Well, three things -

Firstly, I'm not sure an online forum can really be considered a 'group' in the same sense. Online interaction is a completely different beast than personal interaction. Everything is amplified, and because of that I think you'll find that the system is easily driven out of balance by small voices and occasional outliers, regardless of their statistical significance. Given the numbers you just posted, the frustration shown in this thread provides ample evidence of this, does it not?

Secondly - yeah, it only takes two posts equating people to insects for me to head here...

The Oxford scholars, after putting together educational biographies for some 300 known members of violent Islamist groups from 30 countries, concluded that a majority of these Islamist terrorists were not just highly educated, but a startling number of them are engineers. Indeed, according to Gambetta and Hertog, nearly half had studied engineering. A summary of their research in Foreign Policy magazine remarked that "across the Middle East and Southeast Asia, the share of engineers in violent Islamist groups was found to be at least nine times greater than what one might expect, given their proportion of the working male population."

Now, I know you know that I know that you don't mean... Still, in a room full of engineers, let's let not get carried away with this ant thing.

Thirdly, I think that if this voting system works, it will be because of it's flaws rather than it's perfection. It needs to leak a bit around the seams or else you'll end up sucking the life out.

So I've invested too much energy now into something that I don't really give a flip about. I really don't care about who's been done wrong and why and what the excuse is. And I don't care if my rating is 22 or -22 or even the pathetic zero. Though I might be more interested in it if the moderators would respond more to the suggestions people have made, rather than the complaints.

[Dragonfly]

If I could, I'd vote PUP's post up.

After we've been driven to as low as a population of ants, who act on pure pre-programmed instincts, next I'd not be surprized at all if we are compared to machines.

[Absolutelee]

I thought they were misspelling Rants...

[hollus]

One shouldn't gauge whether users are happy or not by counting how many post a negative comment or comparing the happy with the angry posts. Neither in this thread nor in general.
I remember a generic race thread in 2011 when Seb won the race and after the checkered flag, for hours, all posts were like this:
"I am happy"
"Good race, my hero won"
"What a fantastic race, and the best guy today finished ahead"
"The world is sweet and filled with sugar,and the cotton finger man rulezzz"
"Congratulations to the great technical team at Red Bull, a work well done!"

Err, no, actually it went more like:
"Ugh, bloody useless mechanics, they blew my hero's pit stop, he would have gotten Vettel otherwise"
"Senna Schumacher, Hamilton: all sissies"
And so on, you get the drift...

There is such thing as an activation energy, we know it from chemistry:



And in the forum posting world, it looks like this:



The green zone is invisible, it only comes to light if a member makes a random post inspired by the weather, or maybe by a couple of beers; only people in the blue and orange zones are normally bothered to actually make a post, and it is good like that, or we would be flooded! Regarding the voting system, I am obviously in the blue zone, as this post proves, but in general, as in real life, anger is a more powerful action trigger than happiness.

And there you go, you have been degraded not only to ants, but to soul-less molecules. Forum psichohistory* anyone?

*I wonder how many people here get the reference, does it mean I am old?

[i70q7m7ghw]

Okay, I'll state my position…

I’d agree with all your comments. We do need more people voting because then aggregate effects would iron out idiosyncrasies. That brings us onto the most common complaint when members cry foul for an unusual negative vote. We’re all human and in nearly all cases when I enquire about the negative vote the voter has been reasonable from their perspective.

Your exactly right. So why is it so difficult to vote? Everyone should have near unlimited votes in both directions (still limited to 1 vote per post). You shouldn't ever have to investigate a negative vote because it should never matter, it's a single vote. It should take 5-10 negative votes to make any kind of an impact on a post.

[Cam]

Okay, I'll state my position…

I’d agree with all your comments. We do need more people voting because then aggregate effects would iron out idiosyncrasies. That brings us onto the most common complaint when members cry foul for an unusual negative vote. We’re all human and in nearly all cases when I enquire about the negative vote the voter has been reasonable from their perspective.

Your exactly right. So why is it so difficult to vote? Everyone should have near unlimited votes in both directions (still limited to 1 vote per post). You shouldn't ever have to investigate a negative vote because it should never matter, it's a single vote. It should take 5-10 negative votes to make any kind of an impact on a post.

You're asking emotional entities to act without emotion purely on fact alone. Impossible. Diesel, look at our event in the infamous Malaysian race thread. Although (thankfully) no votes were able to be cast - would anyone have wanted too - perhaps not positively, but definitely negatively? There was an attempt to discover an answer and it went pear shaped with trolling answers and a lack of positive posting, to seek the answer, with fact based evidence. It was a disgrace, with myself guilty as well, and was rightly locked. That said, I've attempted to continue to seek the answer via PM - ignored - the answer was given from the team director - no acknowledgement by any of the members who vehemently argued against it. Why? People's feelings got hurt and they responded as such.

Voting is no different. If people were capable of putting emotion aside - none of these events would ever happen, every post would be a positive experience that we can all learn from - but we're human - so maybe it's time to simply let the system be and accept it's not perfect, just like all of us. Try your best and be content that you have have tried your best.

[i70q7m7ghw]

Sorry but I think you've complete missed my point, and I can't really understand what point you are making? Other than to seek me out in a thread I'm active in to issue a "told you so"?

My point is 1 vote is weighted far too heavily. The opinion of 10 people (emotional or whatever) on a post is much more valuable and useful than a single opinion. If a post gets 1 negative vote, how do we know that single person wasn't biased in some way? 9 times out of 10 they were. If it took 10 negative votes to hide a post, now we can be a bit more sure the post is bad. The mods won't have to investigate a single negative vote any more, because it simply won't matter.

There aren't enough votes being cast in either direction for a voting system to work, and that's because it's too restrictive.

[dave kumar]

So why is it so difficult to vote? Everyone should have near unlimited votes in both directions (still limited to 1 vote per post). You shouldn't ever have to investigate a negative vote because it should never matter, it's a single vote. It should take 5-10 negative votes to make any kind of an impact on a post.

Unlimited votes per person (but still limited to 1 vote per post) is probably worth a try. Combine this with widening the number of people that can vote and it would certainly increase voting, which is necessary for the system to gain credibility. My only caveat is that we should not allow unlimited negative votes. In social insects, all (at least all the ones I can think of!) the feedback mechanisms are purely positive feedback (equivalent to up-voting). Negative feedback isn't used at all. From a control systems perspective, having positive and negative feedback is likely to introduce oscillations - instability in the system, unless carefully managed. So I'd be wary of unlimited down-voting, it needs to be severely limited or removed.

Personally, I’d advocate using stars to give a broad brush overview and take the heat out of arguing over individual votes. That would enable us to convey that one thread/member is generally more useful than another thread/member. IMHO that’s the only context for this sort of system. Just like any crowd sourced system, trying to forensically dissect every vote is pointless due the huge variety of personal opinion.

I agree, but in a long thread you still want a way to skim the interesting bits. I think the suggestion of highlighting those posts with more than a certain number of votes without showing the number of votes would be more subtle and therefore less liable to lead to people being offended when they don't receive a vote. It would be nice if the degree of prominence (how much it was highlighted for example) was proportional to the votes it received.

Firstly, I'm not sure an online forum can really be considered a 'group' in the same sense. Online interaction is a completely different beast than personal interaction. Everything is amplified, and because of that I think you'll find that the system is easily driven out of balance by small voices and occasional outliers, regardless of their statistical significance. Given the numbers you just posted, the frustration shown in this thread provides ample evidence of this, does it not?

To address those that worry about us all being reduced to ants in the system, don't lose sight of the aim of the voting system. It just aims to use our collective decision making about what interests us on the forum to highlight the good bits - just as stackoverflow has successfully done with its Q&A forum. It just so happens that the social insects are easier to study than humans and are very good at using collective intelligence to solve hard problems, we can learn from ants without wanting to become them. I for one don't like the idea of living in a colony consisting only of my close relatives.

[Pup]

Unlimited votes per person (but still limited to 1 vote per post) is probably worth a try. Combine this with widening the number of people that can vote and it would certainly increase voting, which is necessary for the system to gain credibility.

This assumes that there is a large body of people who want to vote, but cannot. However, according to Richard, 30% of all up votes and 50% of all the down votes are being made by just 3 people (just 2 in the case of the down votes). Clearly, the problem isn't that not enough people are allowed to vote, but rather that very few people want to.