It’s hard to be a reviewer

Sometimes it’s not easy to be a reviewer of RPG products. If you’ve followed my blog for some time now, you’ve probably noticed that I’ve written quite a few of those in my time. The category “Reviews & First Looks” lists 94 posts. This is quite a lot and in most cases these have been based on a read through of the product in question and not a playtest. And especially this type of review has been criticized recently.

I think most of you remember when John Goodman of Goodman Games called a reviewer an idiot because he didn’t playtest one of Goodman’s adventures but pointed out issues within the module based on a read through. While it’s always a bad idea to react on criticism by calling the reviewer names, it’s also wrong to think only reviews based on a playtest are proper review.

Recently  fellow RPG blogger Dane of War wrote an article that took the same line. While I can understand that he’s annoyed by overly short reviews that appear only hours after a game has been released, I don’t think he’s right. Let me explain why.

Roleplaying games are different from other media because they can be consumed in more than one way. You can’t possible judge on a movie without watching it, and it’s hard to judge an album without listening to it, but there are two ways to consume a roleplaying game: a) you can read it and b) you can play it.

In a way RPGs can be compared to movie scripts. You can of course read the script and write a review based on that or you can watch the movie and judge it by what you’ve seen on the silver screen. Since there’s usually only one movie made using one script, its a good idea to review the whole package.

But things are differently when you look at RPGs. When 100 GMs run a game, you’ll get 100 different experiences. So, if you really want to write a review about the game, do you think including the GM’s abilities into the equation does make the review better? Of course there are things that you might miss in a review based on a read through, but that might happen with the playtest report as well. And especially in the case of reviewers who have played RPGs for a long time, I trust that they can judge a game without having played it.

The second problem is that when you demand reviews based on a playtest only, you will get less reviews and you’ll have to wait much longer. So the new D&D 5th Edition is out, and you are eagerly awaiting the first reviews before you buy it. But alas all the reviewers are still trying to get their D&D 4th Edition campaign finished before they are asking their groups to give 5th Edition a try. If you as a customer (or as a publisher/author) expect reviews close to the release dates, you’re out of luck if you expect the reviewers to playtest all games. It’s just not working that way.

So when you try to please everyone out there you probably have to playtest the game on day one, but in all ways possible, write an in-depth review on the same day and even make sure you focus on the aspects of the game the publisher thinks is most important. That’s an impossible feat. Especially when indie games are concerned it’s often hard to get people to try the game out. Not everyone has a group of players interested in trying out a new game every week.

The title of this post is “It’s hard to be a reviewer”. And this is definitely true. Whatever you do, you will be given crap by people. You like a game, they don’t like, or you review the “wrong” way. I always try to write the best reviews as possible. I read the book thoroughly, take my time, write an in-depth review and try to be fair and objective. What I don’t like is reading on that ‘net the I am “doing it wrong” or that I am an “idiot”.

End of rant.

Michael Wolf is a German games designer and enthusiast best known for his English language role-playing games blog, Stargazer's World, and for creating the free rules-light medieval fantasy adventure game Warrior, Rogue & Mage. He has also worked as an English translator on the German-language Dungeonslayers role-playing game and was part of its editorial team. In addition to his work on Warrior, Rogue & Mage and Dungeonslayers, he has created several self-published games and also performed layout services and published other independent role-playing games such as A Wanderer's Romance, Badass, and the Wyrm System derivative Resolute, Adventurer & Genius, all released through his imprint Stargazer Games. Professionally, he works as a video technician and information technologies specialist. Stargazer's World was started by Michael in August 2008.

13 thoughts on “It’s hard to be a reviewer”

  1. To put up a review, or to publicly express an opinion is to put oneself in the line of fire.

    Personally I believe that it takes a measure of courage for anyone to put his name (or handle) next to a statement of opinion on whether or not a work is good or bad, or could use improvement.

    Reviewers are generally not paid to do this, but publish their work in an effort to help people. Buyers learn the good and the bad, while those that produce the work get further insight into the minds of their audience and consumers.

    Overall, I'm of the opinion that a little respect for everyone, reviewer -or- creator can go a long way to making the internet a much more pleasant place.

  2. I don't think that your manner of review is a problem, especially because you say that it's a read through review. Also alot of us base our purchase on books based on what we've read – the blurb, the first few pages – so I don't think what you do is some major problem.

  3. I agree. I think you need to be upfront with whether or not you played the game you're reviewing, but as long as you're clear about the context of your review, it shoudn't be an issue. I know I'm going to try to be careful about doing just that.

    Good article–

  4. I pretty much gave up on writing proper, serious reviews when Valkyrie stopped being published. I had a bad run of issues that left me feeling reviewing was far too political in the same way that awards can be. Too many people not prepared to here the bad things about their work as well as the good.

    So now the reviews on my blog are less formal and I hope clearly a statement of personal opinion.

  5. As a GM, I will pick up a book and think the following:

    1. Will the players enjoy it?

    2. Will I enjoy it?

    The only way, as a GM, to work that out is to read most of it. If I read all the way through and think it is not right for us then that's a lot of time wasted. This is where reviewers come in – if they review it and describe it as a game our group might enjoy, then I'll get it. This is because, as you rightly mention, a book is consumed first.

    As a reviewer, I only do read through reviews of free RPGs because playing them would not allow me to be consistent. My player group is a very odd bunch – with years of experience playing free games (alright, mostly mine but still). They also don't like miniatures too much and enjoy brutalising holes in game systems. They can be utter bastards when it comes to systems with problems. I know my gaming group is not representative of most groups so doing a playtest review is really not fair to the game. With a read through, I can ensure that I am fair and consistent – my player group would not be.

    That said, I do want to do more playtests of free games but I do not think it's a requirement of a good review.

  6. Good counterpoint, Michael.

    I do agree that yes, it is hard to be a reviewer. Very hard.

    But I stand by the fact that I think it is a disservice to a lot of players without a play-through – in fact, there are a great many micro RPGs that have to be played, because there isn't really that much to read at all.

    But I certainly don't hope that I was singling you out in my post – I wasn't. 🙂

  7. @DaneofWar: I can understand your point but I just don't agree. And I doubt I can convince you of my standpoint. But that's not really necessary, since we respect each other even though we have different views on that issue. But I just had to write something to clarify my stance on the matter. 🙂

  8. So far, somehow, I haven't hit any backlash yet. I'll get the odd correction because I totally misread something, or made a big typo…but no big backlash so far.

    That said, I am now doing more reviews than ever, so that's bound to change…=P

  9. I think it's important for readers to know where the reviewer is coming from so that they know what to expect about the reviewer's taste about the product.

    It helps the reader to choose which reviews he/she should be reading and which he/she knows would just make them mad. So I think the reviewer while doing the review should be upfront about this.

    I've only done several reviews and I sometimes wonder am I doing the review because of how I feel about the product or because I feel obliged to the creator for giving a free review copy.

    To be fair, in the case of RPGs, I think full-time (or someone who has enough time) reviewers should do two reviews; a read-through review and a play-through review. That, IMO, should balance all credibility of the reviewer.

    Just sharing my thoughts and reminds me that I should try to write a post about this, thanks!

  10. I agree with Michael on this… I see Dane of War’s point, and I can understand that some games benefit from a play through, specially the more esoteric kind (and I know this is totally subjective) but as long as there is the disclosure that you read the book, and didn’t spend months playing the game. I think most people who read a review know it’s an opinion, not an absolute…

    Ok did that come out as a rant? Signing off now…


  11. As a publisher, the only time I take issue with a review is when it's clear they've barely even READ the book. I've had a couple reviews of my products where the reviewer got the names of important things wrong, asked for features that were very clearly present already (that no one else had trouble finding), etc.

    If someone can at least give an accurate overview of one of my books, I consider that plenty. If they add their own insights to it, so much the better. And if they like it, that's fantastic!

  12. From Dane's article: "It is an injustice for someone to read through a brand new game book and post their thoughts based on an hour or two read at maximum. That's almost the very definition of "judging a book by its cover"."

    Actually, I'd say it was judging a book by it's *content*, the exact OPPOSITE of that particular idiom.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.