|
c0sm0nautt

Registered: 05/19/08
Posts: 10,303
Loc: The Astral Realm
|
What's the deal with the ratings?
#11913529 - 01/27/10 06:10 PM (14 years, 29 days ago) |
|
|
So I have 60 ratings averaging at 4.6, but the system rounds me down to 4.0? Is there a rating number threshold where 4.6 gets averaged to 5.0, like in normal math?
|
deCypher



Registered: 02/10/08
Posts: 56,232
|
Re: What's the deal with the ratings? [Re: c0sm0nautt]
#11914631 - 01/27/10 08:49 PM (14 years, 28 days ago) |
|
|
The rating system is ranked; a 5 from a person who gives out very little 5's is ranked more than a person who gives a 5 to everyone he can.
-------------------- We are all in the gutter, but some of us are looking at the stars.
 
|
c0sm0nautt

Registered: 05/19/08
Posts: 10,303
Loc: The Astral Realm
|
Re: What's the deal with the ratings? [Re: deCypher]
#11914705 - 01/27/10 08:58 PM (14 years, 28 days ago) |
|
|
So if a person is a dick and gives out a lot of 0's, their 5's will be worth more? 
I don't bother giving out any 0's because I don't see a need for it, but give out 5's when they are due. So why should my 5's be worth less? Seems a little flawed to me.
|
deCypher



Registered: 02/10/08
Posts: 56,232
|
Re: What's the deal with the ratings? [Re: c0sm0nautt]
#11914717 - 01/27/10 08:59 PM (14 years, 28 days ago) |
|
|
I personally think that ratings should remain unweighted.
-------------------- We are all in the gutter, but some of us are looking at the stars.
 
|
Ythan
ᕕ( ᐛ )ᕗ


Registered: 08/08/97
Posts: 18,793
Loc: NY/MA/VT Borderlands
Last seen: 3 minutes, 48 seconds
|
Re: What's the deal with the ratings? [Re: c0sm0nautt]
#11915854 - 01/28/10 12:35 AM (14 years, 28 days ago) |
|
|
Quote:
c0sm0nautt said: I don't bother giving out any 0's because I don't see a need for it, but give out 5's when they are due. So why should my 5's be worth less? Seems a little flawed to me.
That's not exactly how it works... the normalization accounts for mean and standard deviation, so your 5's are not heavily adjusted since most people on the BB give out a lot of 5's. The main effect of the normalization on you personally is to make your bad ratings worth less (ie. more effective), because you leave fewer than average.
Quote:
Ratings left by c0sm0nautt would be worth these values after normalization: Rating of 0 = -2.1642 Rating of 1 = -0.7738 Rating of 2 = 0.6166 Rating of 3 = 2.007 Rating of 4 = 3.3973 Rating of 5 = 4.7877
Similarly, if a member gives out a lot of bad ratings the value of their good ratings may be worth more, but the main effect will be to increase the value of the bad ratings they've left. Eg:
Quote:
Ratings left by sucklesworth would be worth these values after normalization: Rating of 0 = 1.7788 Rating of 1 = 2.5129 Rating of 2 = 3.247 Rating of 3 = 3.9811 Rating of 4 = 4.7152 Rating of 5 = 5.4493
The main thing to remember is that the ratings left by an individual are normalized based on what's typical of the BB, so you can't predict how it will affect someone based on their rating history alone. You can play around with http://www.shroomery.org/forums/ratingtest.php if you like. Normalization is frequently used in this type of situation to account for disparities in subjective judges, and I think overall it has made ratings a more fair and accurate way to gauge a user's reputation at a glance.
|
c0sm0nautt

Registered: 05/19/08
Posts: 10,303
Loc: The Astral Realm
|
Re: What's the deal with the ratings? [Re: Ythan]
#11916406 - 01/28/10 07:10 AM (14 years, 28 days ago) |
|
|
I see, that's actually a pretty good system. I was picturing something a lot more crude.
Now that I know I have the power of the -2, some unlucky person will fall victim to my wrath.
|
waixingren


Registered: 03/14/05
Posts: 2,644
Loc: SW Florida
Last seen: 1 month, 3 days
|
Re: What's the deal with the ratings? [Re: c0sm0nautt]
#11929761 - 01/30/10 01:40 PM (14 years, 26 days ago) |
|
|
lol
--------------------
|
RogerRabbit
Bans for Pleasure


Registered: 03/26/03
Posts: 42,214
Loc: Seattle
Last seen: 1 year, 4 days
|
Re: What's the deal with the ratings? [Re: waixingren]
#11951356 - 02/02/10 10:17 PM (14 years, 22 days ago) |
|
|
I still don't see how this works, and I'm an engineer. 
For example, here's two users from cultivation, both of which only have received +5 ratings, yet one of them has a 4 rating and the other has a 5. They're not bitching, but I just wondered what's up with it.
I noticed the guy with a 4 only had one rating, so I gave him a 5 too, but it still left his overall rating at 4, even though he has two +5s. 
Is a 20% drop normal equalization? RR
-------------------- Download Let's Grow Mushrooms semper in excretia sumus solim profundum variat "I've never had a failed experiment. I've only discovered 10,000 methods which do not work." Thomas Edison
|
Ythan
ᕕ( ᐛ )ᕗ


Registered: 08/08/97
Posts: 18,793
Loc: NY/MA/VT Borderlands
Last seen: 3 minutes, 48 seconds
|
Re: What's the deal with the ratings? [Re: RogerRabbit]
#11951658 - 02/02/10 11:07 PM (14 years, 22 days ago) |
|
|
Heh... when a user leaves a rating, the equation to calculate its normalized value is:
adjusted_rating_value = all_ratings_average + (all_ratings_standard_deviation * (rating_value - user_ratings_average) / user_ratings_standard_deviation)
It's based on the principles explained here.
Then a person's grand score is the average of these normalized ratings, rounded to the nearest whole number.
The two users you're asking about actually work great as an example since they only have a few ratings.
Cension, with a score of five, received a single five-shroom rating from PrimalSoup. PrimalSoup has only left a few ratings so they're not being normalized yet. His five is worth a five (for now).
wygram, with a score of four, received two five-shroom ratings, one from you and one from vinsue.
Quote:
Ratings left by RogerRabbit would be worth these values after normalization: Rating of 0 = -6.32 Rating of 1 = -4.1528 Rating of 2 = -1.9855 Rating of 3 = 0.1818 Rating of 4 = 2.3491 Rating of 5 = 4.5163
Quote:
Ratings left by vinsue would be worth these values after normalization: Rating of 0 = -8.6308 Rating of 1 = -6.0087 Rating of 2 = -3.3865 Rating of 3 = -0.7643 Rating of 4 = 1.8578 Rating of 5 = 4.48
So his rating is (4.5163 + 4.48) / 2 = 4.49815, which is rounded to four. Obviously he's right on the edge and it's likely his next five-shroom rating will push him over the top. I hope this makes sense although I can see how this case was particularly confusing.
|
deCypher



Registered: 02/10/08
Posts: 56,232
|
Re: What's the deal with the ratings? [Re: Ythan]
#11951675 - 02/02/10 11:09 PM (14 years, 22 days ago) |
|
|
Why are ratings normalized in the first place?
-------------------- We are all in the gutter, but some of us are looking at the stars.
 
|
Ythan
ᕕ( ᐛ )ᕗ


Registered: 08/08/97
Posts: 18,793
Loc: NY/MA/VT Borderlands
Last seen: 3 minutes, 48 seconds
|
Re: What's the deal with the ratings? [Re: deCypher]
#11951732 - 02/02/10 11:22 PM (14 years, 22 days ago) |
|
|
There are two reasons: it was primarily conceived to lessen the impact of rating rapes, but it also makes ratings more accurate and consistent between members which I consider desirable in and of itself. Consider a hypothetical case where we have some members who only give out zero-shroom ratings time after time, and others who only give out five-shroom ratings. Is it really more "fair" or "accurate" to give these ratings full weight, or for them to trend towards the average value of a rating on the BB? I guess reasonable people could disagree but I'd argue the latter. Given this and its anti-abuse properties it seemed like a clear win. Although I acknowledge it's more complicated and confusing than the previous system.
|
deCypher



Registered: 02/10/08
Posts: 56,232
|
Re: What's the deal with the ratings? [Re: Ythan]
#11951751 - 02/02/10 11:24 PM (14 years, 22 days ago) |
|
|
I mean, it's definitely counter-intuitive. Ratings rapes do exist but does their presence mean that we have to make one's rating score needlessly obscure?
-------------------- We are all in the gutter, but some of us are looking at the stars.
 
|
deCypher



Registered: 02/10/08
Posts: 56,232
|
Re: What's the deal with the ratings? [Re: deCypher]
#11951762 - 02/02/10 11:25 PM (14 years, 22 days ago) |
|
|
The mode might be a good addition to the mean in this case.
-------------------- We are all in the gutter, but some of us are looking at the stars.
 
|
Ythan
ᕕ( ᐛ )ᕗ


Registered: 08/08/97
Posts: 18,793
Loc: NY/MA/VT Borderlands
Last seen: 3 minutes, 48 seconds
|
Re: What's the deal with the ratings? [Re: deCypher]
#11951853 - 02/02/10 11:47 PM (14 years, 22 days ago) |
|
|
Not needlessly obscure, just obscure. And I would answer yes. A man can only take so much bitching.
I never took a statistics class so I went with calculating a standard score since it seems to be the approach used in similar situations. But I'm open to suggestions, if you can explain why your method is better than the one used by statisticians (or why the one I'm using is wrong in this context).
|
deCypher



Registered: 02/10/08
Posts: 56,232
|
Re: What's the deal with the ratings? [Re: Ythan]
#11951861 - 02/02/10 11:49 PM (14 years, 22 days ago) |
|
|
Well it seems like an OTD crew (or whoever decides to ratings rape) could never outweigh the majority of other Shroomery users when it comes to determining the mode of the ratings received for a particular user. Thus this would possibly be more accurate than using a mean, even if weighted.
-------------------- We are all in the gutter, but some of us are looking at the stars.
 
|
Ythan
ᕕ( ᐛ )ᕗ


Registered: 08/08/97
Posts: 18,793
Loc: NY/MA/VT Borderlands
Last seen: 3 minutes, 48 seconds
|
Re: What's the deal with the ratings? [Re: deCypher]
#11951958 - 02/03/10 12:11 AM (14 years, 22 days ago) |
|
|
Well right now the current score of a user receiving a rating is not taken into account during normalization. When I said 'user_ratings_average' and 'user_ratings_standard_deviation' above, I meant for the person who's leaving the rating. I'm not exactly sure how you'd incorporate the recipient's mode into the normalized value or even if it's desirable. It seems like rather than minimizing the effects of abnormal ratings it would serve to amplify them, at least in the short-term. And I'm not sure how it would affect the resulting 'accuracy' of the normalized scores. But remember you've got to explain this to to me like a guy who last studied math as a high-school sophomore over 10 years ago. I'll defer to the expertise of others if you can sell me on your idea. Providing an actual equation I can plug in to the script might help.
|
deCypher



Registered: 02/10/08
Posts: 56,232
|
Re: What's the deal with the ratings? [Re: Ythan]
#11951968 - 02/03/10 12:14 AM (14 years, 22 days ago) |
|
|
The mode = the most frequent rating given to a particular user. Since the rest of the site outweighs any potential ratings rapage I would anticipate that the mode would most accurately reflect a particular's user's 'general' rating.
-------------------- We are all in the gutter, but some of us are looking at the stars.
 
|
Ythan
ᕕ( ᐛ )ᕗ


Registered: 08/08/97
Posts: 18,793
Loc: NY/MA/VT Borderlands
Last seen: 3 minutes, 48 seconds
|
Re: What's the deal with the ratings? [Re: deCypher]
#11952089 - 02/03/10 12:42 AM (14 years, 22 days ago) |
|
|
Heh... no, I know what a mode is but I don't see how you'd use it to come up with a more accurate normalized rating. You have to be specific about what technique you would use. You can't just tell me "use the mode", I don't know of a method for normalization which incorporates the mode and I don't know how to come up with one which has any statistical basis behind it.
|
deCypher



Registered: 02/10/08
Posts: 56,232
|
Re: What's the deal with the ratings? [Re: Ythan]
#11952103 - 02/03/10 12:44 AM (14 years, 22 days ago) |
|
|
I guess I'm unsure why normalization is required in the first place... if the non-normalized mode of a user's ratings creates an accurate reflection of the content of his posts then why not legitimize it?
Sorry, I'm very high at the moment.
-------------------- We are all in the gutter, but some of us are looking at the stars.
 
|
waixingren


Registered: 03/14/05
Posts: 2,644
Loc: SW Florida
Last seen: 1 month, 3 days
|
Re: What's the deal with the ratings? [Re: deCypher]
#11952127 - 02/03/10 12:49 AM (14 years, 22 days ago) |
|
|
i like the way deCypher's idea is going. i'm trying to understand how it could be implemented though.
if the rating being given to the receiver is equal to the mode rating of the receiver then it is just averaged in. if the rating being given does not equal the mode of the receiver then it is normalized based on the current system. the resulting normalized mean and the un-normalized mode are then averaged out.
does that make sense?
--------------------
|
|