Other Software > Developer's Corner

How NOT to Sort by Average Rating

(1/1)

Deozaan:
I came across an article today that explains some flaws in commonly used sorting methods, with examples of why they're wrong, and then proposes a nice solution for more accurately sorting the truly higher rated stuff on top.

PROBLEM: You are a web programmer. You have users. Your users rate stuff on your site. You want to put the highest-rated stuff at the top and lowest-rated at the bottom. You need some sort of "score" to sort by.

WRONG SOLUTION #1: Score = (Positive ratings) - (Negative ratings)

Why it is wrong: Suppose one item has 600 positive ratings and 400 negative ratings: 60% positive. Suppose item two has 5,500 positive ratings and 4,500 negative ratings: 55% positive. This algorithm puts item two (score = 1000, but only 55% positive) above item one (score = 200, and 60% positive). WRONG.

Sites that make this mistake: Urban Dictionary



WRONG SOLUTION #2: Score = Average rating = (Positive ratings) / (Total ratings)

Why it is wrong: Average rating works fine if you always have a ton of ratings, but suppose item 1 has 2 positive ratings and 0 negative ratings. Suppose item 2 has 100 positive ratings and 1 negative rating. This algorithm puts item two (tons of positive ratings) below item one (very few positive ratings). WRONG.

Sites that make this mistake: Amazon.com
-http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
--- End quote ---

The article goes on to propose a better method of sorting things by their rating. A method which has even been endorsed by Randall Munroe of xkcd fame, so you know it's good. ;)

Anyway, be sure to read the full article here:
http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

mouser:
Interesting read, thanks for sharing.  :up:

ewemoa:
Does seem interesting -- not so easy to get through and understand here :)


Have a suspicion that there is an assumption of ratings having a certain distribution (given the mention of "95%") -- anyone who understands, can you confirm?

Navigation

[0] Message Index