Judging a book by its content

I recently wrote a couple of reviews for this very blog without having playtested it. That kind of review is pretty much common for most RPG reviewers, but also criticized by some readers and sometimes even publishers. I actually wanted to write a post in which I share my thoughts on the matter, but then I realized I have already written that post back in 2010. What follows is a repost of that 2010 article called “It’s hard to be a reviewer”.

“It’s hard to be a reviewer” by Michael Wolf

(this post first appeared on Stargazer’s World on July 27th 2010)

Sometimes it’s not easy to be a reviewer of RPG products. If you’ve followed my blog for some time now, you’ve probably noticed that I’ve written quite a few of those in my time. The category “Reviews & First Looks” lists 94 posts. This is quite a lot and in most cases these have been based on a read through of the product in question and not a playtest. And especially this type of review has been criticized recently.

I think most of you remember when John Goodman of Goodman Games called a reviewer an idiot because he didn’t playtest one of Goodman’s adventures but pointed out issues within the module based on a read through. While it’s always a bad idea to react on criticism by calling the reviewer names, it’s also wrong to think only reviews based on a playtest are proper review.

Recently  fellow RPG blogger Dane of War wrote an article (Editor’s note: this article is not available any more) that took the same line. While I can understand that he’s annoyed by overly short reviews that appear only hours after a game has been released, I don’t think he’s right. Let me explain why.

>Roleplaying games are different from other media because they can be consumed in more than one way. You can’t possible judge on a movie without watching it, and it’s hard to judge an album without listening to it, but there are two ways to consume a roleplaying game: a) you can read it and b) you can play it.

In a way RPGs can be compared to movie scripts. You can of course read the script and write a review based on that or you can watch the movie and judge it by what you’ve seen on the silver screen. Since there’s usually only one movie made using one script, its a good idea to review the whole package.

But things are differently when you look at RPGs. When 100 GMs run a game, you’ll get 100 different experiences. So, if you really want to write a review about the game, do you think including the GM’s abilities into the equation does make the review better? Of course there are things that you might miss in a review based on a read through, but that might happen with the playtest report as well. And especially in the case of reviewers who have played RPGs for a long time, I trust that they can judge a game without having played it.

The second problem is that when you demand reviews based on a playtest only, you will get less reviews and you’ll have to wait much longer. So the new D&D 5th Edition is out, and you are eagerly awaiting the first reviews before you buy it. But alas all the reviewers are still trying to get their D&D 4th Edition campaign finished before they are asking their groups to give 5th Edition a try. If you as a customer (or as a publisher/author) expect reviews close to the release dates, you’re out of luck if you expect the reviewers to playtest all games. It’s just not working that way.

So when you try to please everyone out there you probably have to playtest the game on day one, but in all ways possible, write an in-depth review on the same day and even make sure you focus on the aspects of the game the publisher thinks is most important. That’s an impossible feat. Especially when indie games are concerned it’s often hard to get people to try the game out. Not everyone has a group of players interested in trying out a new game every week.

The title of this post is “It’s hard to be a reviewer”. And this is definitely true. Whatever you do, you will be given crap by people. You like a game, they don’t like, or you review the “wrong” way. I always try to write the best reviews as possible. I read the book thoroughly, take my time, write an in-depth review and try to be fair and objective. What I don’t like is reading on that ‘net the I am “doing it wrong” or that I am an “idiot”.

Editor’s note: The title of this post is a quote from a comment written by the user Jiggen under the original article in 2010.