Does usage of thumbs up/down ratings as social proof actually provide the right feedback?

Let me stay aside from the purpose of this change. There is some business decision behind it, for sure. But regarding the scale itself I am sure it is no good for the accuracy and telling good from better.

Thumb up/down does not allow that because it lacks proper resolution. It does not allow to register “good” and “better” at the very beginning. It is a simple, -1/1 measure. It is, indeed, anchored in the world of quality, but in fact being mostly qualitative.

At the same time, I strongly believe that a 1-5 (or 0-5) scale is just as accurate for that as it should be. It allows me to say a lot about a movie that -/+ does not:

  • “Dude, that was a total massacre, don’t even think about seeing this film!”
  • “It was bad, but not that bad.”
  • “Well, just another movie like a lot of them. A good one to kill time.”
  • “Wow, it was actually quite good!”
  • “That was a masterpiece!”

As a result, thumbs down/up system throws all the masterpieces and quite good movies into the same bin. Moreover, putting more weight on the number of voters, it makes those exquisite niche movies be outscored by that family comedy that just hit the screens. It dangerously pushes the measure from “how good” towards “how many”. And this is bad for both rating and sensing the quality.

EDIT:

So, going further. I think that introducing this kind of measure deteriorates the matching significantly.

  • For a 1-5 rating even one or two votes can shape my opinion somehow, and even if I do not know those users’ preferences, I would give this movie a try.

  • With a thumb up/down mechanism, I need to see e.g. a hundred of them to build up my interest.

In other words, I will get hooked on what is popular, not on what is good. And unless “just an average regular normal guy”, this will not match my preferences.

Is there a way out of this? Well, there is! Should my profile be put against the profiles of other users, so that the preference match is at the user profile level a thumb up/thumb down mechanism could be just enough. In other words, I believe it would be possible to match, to some extent, a movie with my preferences based on a combination of a simple thumb up/thumb down system and a profile matching system, that would recommend me movies “upthumbed” by people with similar taste. Unfortunately, for that, a quite elaborate user preferences profile should be built first. As I am not a Netflix user, I cannot tell if it is somewhere there below.

This is something that one of our local movie services, Filmweb, once introduced. I took a part in a long survey that tested my preferences which should result in presenting better suggestions to me. It is a pity I have never tried to use it after that, though, but I still think it could be a good idea. But even then, it is more about recommendations, not rating.