This post will compare the rating systems used to evaluate individual online sellers. These are often of critical importance to buyers in order to avoid scammers online. Usually, the higher the number of ratings of a seller, and the higher those ratings, the more likely they will be legitimate. It sucks to buy an Xbox off eBay and have it just be the box. You would likely respond with a negative post, thumbs down, scammer, 0 stars. We use the ratings systems to avoid this, but how accurate are they?

The most commonly seen rating on ebay is a compilation of binary variables. That is, a good rating or a bad rating. These are tallied up and given a percentage of good ratings over total ratings. A seller of 96% for instance would only have 4% bad ratings. This is further made accessible by a star rating system which indicates how many ratings have been given. Different colour stars indicate a different number of ratings in total. This is good for data transparency but is hard to decode, as the colours follow a pretty random hierarchy. Further you can see individual’s 5 star ratings and comments for a more detailed view of some experiences with the seller.

Google merchant uses a more holistic approach to calculating seller scores. It uses aggregate totals from snippets of customer reviews from 3rd party sites and also Google Checkout. They re-scale each review they find to a 1-5 scale. The problem is they don’t give any information on how they tally this score, just that “trust them, they do it accurately”. Here is a snippet from their official seller score page.

“We calculate Seller Ratings using a variety of signals beyond just the arithmetic mean in order to make sure Seller Ratings reflect not only the raw quantity of review scores, but also how representative and high-quality the reviews are. We’re constantly refining how we use those signals to give our users as helpful an overview as possible.”

eBay’s scores may reflect some polarization. Chances are, if you didn’t have any major problems with your experience, you would give a moderate review. This would likely be counted as a ‘thumbs up’ and therefore added to that total. As a result, most sellers on eBay who do so fairly regularly get scores in the high 90%’s. The degree of variation doesn’t neccessarily reflect the variation you might have in experiences with different sellers.

Google Merchant on the other hand, is hard to trust. They simply say that they throw it all into one big accurate secret calculation and ta-da! your super precise aggregation of the entire internet has created this score out of 5 for how reliable this seller is. It is good that they get these ratings from a bunch of different sites, but some transparency as to how the total is calculated is needed.

Posted: 02|12|12 at 2:41 pm. Filed under: Uncategorized.
Comment | Trackback