Pooling Reviews - A simple solution to a complex issue
Today I’m going to write about reviews, the problems with them and a simple solution. I believe that there are many issues with today’s reviews and that this could be rectified by pooling reviews. I’m not talking about how Metacritic.com does it; I’m talking about each site/magazine pooling reviews from multiple people on their site or in the magazine.
So let’s start off by asking what’s so wrong with the way reviews are done now and why do they need fixin’? Reviews are an opinion of an individual person. That person is generally supposed to go at the game in a non-biased way and look at it objectively. That is impossible. Everyone has a bias, it may not be strong but it’s there. There is no one out there that can honestly say they like all genres equally and the same. They can try to be objective, but the sub-conscious feelings can not be disposed of, they are sub-conscious and not on purpose (for the most part). So some people out there would say to have a person who is fond of that genre review the game. That’s not fair or un-biased either. They would be looking at the game in a more positive way, but that is not fair to us, the consumer looking for a true review. No one wants to feel placated, like the reviewer is praising the game because that’s what we want. In the same respect we would not want someone to unjustifiably crush our game with a bad review because they didn’t understand or appreciated the whole thing.
Another issue with reviews of today is the increasing feeling of low credibility of the reviews and reviewers. Reviewers are being called out on inadequacies of their review now more than ever. Whether it is because they don’t appreciate the genre (Dynasty Warriors), want to appeal to the larger market (COD), or for any other reason. Sometimes they are un-justified, but often there are some truths to the complaints. Is it because the ever increasing scrutiny of the internet? Is it because we are allowing the flawed opinions skate through as if they were true? Or is it something else? It’s hard to say really, but probably a little of everything becomes fuel for this slow burning fire.
Some might be quick to point out that sites and magazines that do reviews often have a second opinion of the game or review just a portion at a time. That is good for us as the consumer, but no so much for the people who make the game. Metacritic only allots one review slot per site, so that second review or second half doesn’t even get counted on their aggregated score. For those who say Metacritic doesn’t matter, while it may not reflect the opinions of most, it does have its importance. Often a developers pay is determined by how well a game scores, and Metacritic is the fairest way to determine how it scored as it pools the reviews of many into one single number. So a developer’s livelihood is sometimes dependent on how well the gaming journalist community feels about their game. So I ask, don’t we owe it to ourselves and these developers to make sure the reviews making it to Metacritic and to the consumers to be are fair, unbiased, and truly represent the game as a whole package?
So I propose that gaming sites and magazines start pooling reviews of three or more people from their establishment and combine them into a larger, more accurate review. It’s really the fairest way to really get the best opinion of the game. Instead of having reviews that swing in a poor direction or an overly positive one, we would have a more balanced and true review that gives us multiple opinions rolled into one convenient location. This would also help alleviate the problems with Metacritic scores as well as the score given to each game would be inherently more accurate reflection of the experience. Reviews have basically been going on the same way since movies first game out and not much has changed. Isn’t it time for an overhaul of how things are done? It seems, to me at least, that the gaming community would welcome such change with open arms. This could possibly help sites losing credibility regain some too.
That is obviously my opinion on the matter. Let me know in the comments below how you feel about this subject. Maybe there are some inadequacies in my review of the reviewers. Lets keep it clean though.