Reviewing The Reviews

10 years ago | Eli James (Moderator)

Thought I'd get a feedback loop set up for all of WFG's reviews (editor and member alike). Here's the deal - you post up (and link!) any review that you like, or a review that you don't, and you tell us why. It can be by a member, editor, whatever (speaking for myself, you can kill me if I've written a bad one) and trust me, I won't bite =).

A couple of guidelines:
1, Be honest. The whole WFG community will probably be learning from what you say here, so please tell us what you think.

2, Be polite. You know that guy. Don't be that guy.

3, Carrot over stick. Point out reviews you've enjoyed, not just the ones that've irked you off, and plug reviewers you think are worthy of recognition. I'm sure we can all do with better reviews, and praising people for their efforts is a good way of getting there.

4, Learn! As I said, we can all be better reviewers.

5, Teach! If you have issues with a review, please post up suggestions and alternatives. It's much easier to see your argument if you demonstrate how it should've been done.

Anyone up to this?

Read responses...

Page: 128

Responses

  1. Eli James (Moderator)

    Posted 10 years ago

    Allow me to get the ball rolling:

    I think the member reviewer that I've enjoyed most recently is Miladysa. She's very honest with what she says, and everytime I read one of her reviews (of which there are many), I get the feeling that this is somebody who absolutely loves reading web fiction. Not particularly sure about the rating bias she has (most of her reviews are 4-5 stars, which is scary, but I'd like to think that's because she reviews what she likes) or the fact that some of the reviews she does are vague. But there's this very ... bubbly tone to what she writes, and I like that. I really, really like that.

    Not all reviews are quite so charming. I was doing one on The Tom Drake Experience not too long ago, and there was this one review that irked me off to no end:

    Young urban professionals, the Esquire generation, can relate to The Tom Drake Experience. It’s somewhat like Fight Club without the blood and gore. The influence of Tom Chiarella is palpable.

    And it was a five star review. I mean, what does that tell me, as a reader? If I was new to WFG, and I stumbled onto this page by accident, would I be impressed by a three sentence five star review? Hrmmph. =(

  2. Miladysa (Member)

    Posted 10 years ago

    I think this a brilliant idea ejames :D

    I will BBL to add my bit about reviews but for the time being here is my response to the points you have raised regarding my own:

    1. Thanks :D

    2. I am honest in all my reviews

    3. I LOVE web fiction!!!!

    4. Web Fiction writing and reading is the way I relax and I prefer to make it as pleasurable an experience as I possibly can.

    So far I have probably read more than four times as many sites as I have taken the time to review or recommend. I have only committed my (much treasured free) time to reviewing the ones I thoroughly enjoy/ed and can be really be positive about.

    There are other sites that I am enjoying and intend to review when there is a bit more of the story posted. This is because I have noticed many have started well, received great reviews in some cases and have then been abandoned (I'm not talking about the writer taking a break - heaven knows we are entitled too!) and as a reader you are left mid air.

    I realise that some people may say that I should take the time to review the sites I did not enjoy and give the writer some feedback and an opportunity to address the issues I think are there. My answer to that is that in most cases these sites have already received a review pointing out the same things and a. I do not wish to rub salt into an open wound and b. I can add no more than what has already been said.

    Finally, I have noticed that many writers (including myself)have taken on board the feedback given in their reviews and edited their stories and/or adressed the issues within their sites. In such cases I have updated my reviews accordingly and have raised my star ratings. I would love to see some of the Editors do the same thing where warranted.

    Once again, many thanks for starting this thread!

  3. G.S. Williams (Member)

    Posted 10 years ago

    I don't know if I need to link to them, exactly, because I want to broadly say that I like Eli's reviews. I've been an admirer of his clear writing style since Novelr, and it's been a privilege to write some articles there and work with him here. I find his reviews are deeply thoughtful, and insightful. I measure my best reviews by his standard, as I think Eli tries to both educate about literature in general, and also, I think he tries to help explore the possibilities for fiction specifically offered by the Web.

    The only other editor that I think goes into a similar depth in a review might be Chris, even though everyone tries to be as honest and informative as possible. But the three of us seem to attempt essays every once in a while. :)

  4. Sora (Member)

    Posted 10 years ago

    I like Miladysa's and Donna Siranni's reviews. I can't think of any reviews in particular, but I have the reviews in an RSS feed so I enjoy reading them and checking out the new listings.

    Honestly, I'm a little bit afraid of the editors. A few of them seemed to be a little biased (I'm not going to say any names) as a lot of reviews start off stating "I'm not really into this genre, but...".

    I'm updating a few of my reviews to have a summary of the story, my likes, dislikes, and my overall. I find that this works better (I don't know why I really didn't think of this before.)

    At any rate, I was curious. How do the editors go about choosing which stories they review?

  5. Jim Zoetewey (Moderator)

    Posted 10 years ago

    Personally, I mainly go with what sounds interesting. That being said, I also look at whether something is really old and hasn't gotten reviewed yet. In that case I'll do it if no one else seems interested (whether it sounds interesting or not).

    I suspect that other editors do the same.

  6. Samazing (Member)

    Posted 10 years ago

    I've only been here a short time, but here's one review that I'm having difficulty with which I found when I started reading "Street"... http://webfictionguide.com/members/snitschke/street/

    The reviewer's opinions are generally favorable with what she makes clear are some misgivings, but the score and her conclusion do not match up at all with the body of the review. Apparently she revised the score, because she had allowed her personal preferences to get in the way, which strikes me as practically heresy. If you're going to review something, then you have to review it from where you stand, nowhere else. Now, I get the feeling that there is definitely something I missed, and if I'm opening up an old can of worms then apologize, but her words don't seem to argue for a nearly perfect grade. As if she had an opinion about the work, but changed it to fit...expectations.

    And I'm not trying to make a comment on the work she reviewed here, or slight Sonja Nitschke in any way, but this struck me as a pretty large discrepancy right out of the gate. I haven't yet had the chance to read many other reviews and only written two myself, so I can't comment any further on anything else yet, so unfortunately I have no positives to balance out my observation (if it bears out as a negative, that is).

  7. Eli James (Moderator)

    Posted 10 years ago

    @Miladysa: You're welcomed. =)

    On the positive skew in member reviews: it's actually quite worrying for us editors, to tell you the truth - if everyone were to post up only 4 stars and above, that wouldn't make WFG very useful for readers, no? I do understand your decision to only review the good ones, and I appreciate your willingness to revise your scores upwards should the writer improve. But we're also starting to wonder just how problematic this might be in the future - there's often a huge dissonance between editor and member reviews on less popular works. It's as if the member reviews that come after an editor are purposely polarized as to try to counter-act the editor in question. =( Also, reviews from the review exchange thread tend to be positive, and even if they were negative the scores are very rarely below 3. Is this because of the nature of the exchange? Or is it because the members are chums? I'm not really sure. Anyone has any other opinions on this trend?

    And yes, on the upward revision of scores, we tend to do that if we have a bit of breathing room with the unreviewed listings. I'm on assignment from Chris, for instance, to review the listings that have too little editorial reviews.

    @Gavin: Thanks Gavin! This is rather ironic, considering that I give your reviews more weightage when considering what works to read next. And, yes! We do have a tendency to write essays to justify our scores. I think we have a tendency to write too much.

    @Sora:

    A few of them seemed to be a little biased (I'm not going to say any names) as a lot of reviews start off stating "I'm not really into this genre, but...".

    Any suggestions or alternatives to this? Do explain, please.

    I've generally thought that it was only right to disclose any bias you might have against (or for) a genre at the outset, and then proceed to attempt to rate the review based on normal expectations of said genre. But I may be mistaken, of course. What you've said about the editors being scary does seem to apply across the board, however - we've had reports of people being afraid to list with us. Is there any reason you may think for this? The tone of reviews, perhaps?

    @Samazing: I'll let Sonja respond to that herself, but for the record, my next assignment is Street, and I'll probably be writing another essay.

  8. Ryan A. Span (Member)

    Posted 10 years ago

    Eli: I reckon it's not a bad thing that the editorial part of the site puts a bit of pressure on writers -- it's what makes the WFG different from every other web fiction listing site. Basically, if you submit here then you'd better bring the goods.

    The main thing people seem to be scared about, at least to my mind, is that a bad review here could put off readers that might've given it a chance otherwise. They're wrong, of course, and that's a very counterproductive attitude for a writer, but it's a very easy trap to fall into.

    Sora has a point about the reviewer bias, though. I think it'd go a lot smoother if you matched up editors and stories with their own preferences rather than just selecting at random. One possible system is to give stories two editor reviews to start with, one by someone who likes the genre and another by someone who doesn't. That would broadly investigate the story's appeal to any passing reader whether it's something they'd normally pick up or not.

    As for Sonja's Street review, I'm not going to comment on it myself. I expect Sonja can speak for herself.

    I hope it's a good essay, by the way. ;)

    Regards,
    Ryan

  9. Miladysa (Member)

    Posted 10 years ago

    @ejames - As a reader I am only going to read what appeals to me - I'm not reviewing because it is expected of me as an editor.

    If I am not enjoying a story I abandon it after a fair whack - I'm not then going to waste my precious free time reviewing it when I could be working on my own story. Neither am I going to bother reviewing many of the editor sites which have already received numerous reviews from other editors unless I really rate the story. When I really rate a story I then take the time to write a review and rate it highly.
    Where I have reviewed a site it is guaranteed that my review is based on the ENTIRE story at the date of that review - not just five chapters. Where I have taken the time to review a story with only a few chapters posted you can bet that I am really looking forward to reading more from that writer.

    As for possible bias in the review exchange I personally find that idea insulting. Perhaps our reviews are more positive because we have volunteered to review and do not feel obligated to do so?

    The WFG claims to be reader rather than writer orientated - if editor reviews are so far removed from reader reviews then I think the editors need to look at their review rating. When I write a review the gist of it is based on the enjoyment I gained from the story and how enjoyable I think it will be to another reader.

    In my opinion some of the editor reviews can border on nasty - I'm not going to single anyone out because I do not think there is any one person in particular.

    "It's as if the member reviews that come after an editor are purposely polarized as to try to counter-act the editor in question."

    After reading some reviews I wonder if the editors have actually read the same story as myself and from comments on other sites I know others have the same thoughts too. Could this be because some of you are not taking the time to read through the whole story? Possibly.

    I know your time is precious like the rest of us but you have chosen to be in that position and if you are going to do reviews I think you should read through what is available at the time of that review. Personally I do not think it would be a bad idea if a story with only a few chapters was left to develop before they receive an editor review.

    Finally - my favourite editor is Chris Porrier - you can tell from his reviews that he has taken the time to read through the story, put some thought into constructive rather than destructive criticism and he is always fair. I do not think Chris has frightened anyone away from here but I am sure some of the editors have.

  10. Miladysa (Member)

    Posted 10 years ago

    I just wanted to add that typos, spelling, grammatical and other such errors do not in any way distract from my enjoyment if the story grips me. I am sure that some WFG members will vouch for the fact that where I have noticed such oversights I have taken the time to mention them in the comments section or sent an email to the author. I have also entered into lengthier emails with writers giving additional feedback where requested.

    I am not sure if anyone here considers me ‘a chum’ but I can honestly say I have never rated a site on a ‘chum basis’.

  11. Eli James (Moderator)

    Posted 10 years ago

    @Ryan: Good point about Sora's idea. And as for the essay, I'm not saying either way - my only job is to be honest. =)

    @Miladysa: Thank you for your feedback - what you've said will allow us to improve the site, and the user-experience. However, it appears to me that you've taken what I said personally, and I apologize if I've given the impression that I was singling you out (and how could I, considering I enjoy your reviews?).

    I struggled with the idea of not finishing a whole work before doing a review, but you have to understand the pressure and the reality of being an editor at WFG. We have to cover and evaluate all new listings to ensure that our users are able to find things they'll like to read; so we work under a deadline and a steady backlog of unreviewed, unrated new works. I've written about the process in Novelr here, and allow me to quote the related portion:

    I’ve had some time to mull over that system, and I have to conclude that whatever system we use is often created out of necessity rather than personal preference. WFG posts up to 3 new listings a day, many of them novel length, and to keep up with all these listings is nigh on impossible if we’re to read through everything. Part of our problem is that most of these works don’t even deserve a review in traditional critical establishments - they get filtered and are left in the slush piles of publishers. There’s actually an agency that provides professional assessments of unsolicited manuscripts, but there’s a catch: you have to pay good money for that service (and they will read everything, and pass it on to a publisher if they think it’s worth a shot). WFG editors aren’t paid anything, and so we have to make do with what we have.

    My view on reviewing systems (and we all have different ones) is that I will read as much as is needed for me to write a fair review. Some works, the good ones, I’ll read through from first to last. The lousier ones, however, I’ll pass after 1 or 3 or 5 chapters. My job is to tell you if it’s worth your time, and the best way to do that is if I’m honest. Forcing the editors to read through everything, particularly if that story is bad, does nothing for either reader or editor: you’re not likely to appreciate a work for its themes or ending if you can’t get past the first few chapters.

    An alternative would be greatly appreciated, but at the moment this is the best we've got. Trust me, we're trying hard.

    As for possible bias in the review exchange I personally find that idea insulting. Perhaps our reviews are more positive because we have volunteered to review and do not feel obligated to do so?

    Do you suggest, then, that we close an eye to the possibility of bias? I believe, Miladysa, that it's fully within our scope to be worried about the quality of member reviews, and about how we're going to effectively leverage the community to help the a new reader find good things to read. We're constantly trying to overcome and compensate for bias, or for gaming the system. In fact, WFG started in part from an analysis I did on the fallacy of small crowds in Pages Unbound. For instance, when I say that a review is purposely polarized as to counteract an editorial rating, I mean this and this. Look at those and tell me that our concerns are unfounded. Chris built WFG from the ground up trying to prevent fanboys from gaming the system and shooting a sub-par work to the top of the listings. This is a challenge that we've faced on a day-to-day basis, working here. But I believe it is a necessary challenge, if we want online fiction to be widely read.

    I agree with you that WFG is reader oriented, not writer oriented (though that issue is currently heavily debated by the editors). But let's set aside that debate for a moment and accept that I agree with you: WFG is reader oriented in the sense that we try our best to help readers find good things, new things to read. This is our editors' focus. Your point about how editors need to look at their ratings if they're dissonant from the member's ratings misses this point - our job is to tell you if it's worth your time. Members, however do not have this obligation, and they review for different reasons. Some review because they know and like the author. Others just want to share what they've found. Still others do it because of a review exchange, or maybe they do it because they're not happy with the scores an editor has given. And is it not logical that when you do a review exchange, you don't want to be too harsh, that there is a psychological factor that prevents you from being too negative with the writer you're exchanging a review with? This is the chum factor that I speak of. Mind, we're supportive of the review exchange, because we think it's healthy. But we do have questions, and we're constantly thinking about the possible downsides, because it's our jobs to do so.

    I know your time is precious like the rest of us but you have chosen to be in that position and if you are going to do reviews I think you should read through what is available at the time of that review. Personally I do not think it would be a bad idea if a story with only a few chapters was left to develop before they receive an editor review.

    Please, consider your words. I feel uncomfortably taken for granted when you put it like that. I too, have family, and all of us have our daily jobs and our real-world responsibilities. We do this only because we're passionate about the cause of online fiction. As for the part about editors witholding from reviewing new works, we've recently had a discussion about that in the private editor forums, and the very suggestion you've proposed have been in effect for two weeks now.

    In my opinion some of the editor reviews can border on nasty - I'm not going to single anyone out because I do not think there is any one person in particular.

    Do explain, and give me your thoughts, and concrete examples. The whole point of this thread is to overcome some of the perceptions people have about editorial reviews. We're trying to get to the bottom of this and solve it. Is it our tone? Is it the way we handle new writers? How can we improve? Help us.

  12. Sonja Nitschke (Member)

    Posted 10 years ago

    @Samazing

    I wrote the review when WFG was still growing up. Editors roles were still being hammered out, WFG's purposes were still trying to be defined, even the meaning of the Star Ratings was being questioned and changed. The ratings didn't mean then what they mean now.

    Shortly after I wrote the review, there was talk that some of us were coming across reviewing from not quite the right direction. I considered everything that was said in that thread and realized that, as an editor, I did not have the luxury of reviewing from my own tastes (which is what I had done before). You said,

    "Apparently she revised the score, because she had allowed her personal preferences to get in the way, which strikes me as practically heresy. If you're going to review something, then you have to review it from where you stand, nowhere else."

    However, I stand for a lot of things: I stand for good writing, three dimensional characters, and plot pacing. All of the above is what I base my star rating. I let people know my personal opinion, but a star rating is supposed to be a flag that helps people decide if they want to read it or not -- not as an echo of my personal opinion.

    Am I wrong to think that, as an editor, my opinion shouldn't be reflected in the star rating? I don't know. We editors get a lot of mixed messages. People have implied that they are afraid to list with us. People have said that we are biased in some form or another. Is it really a surprise that I attempt to keep my unmeasurable, and oftimes illogical personal preferences away from the very measurable, logical algorithms which incorporate the star ratings into the functions of WFG?

    I've changed my score of Street three times -- each time after the site was changed or the ratings redefined or something like that. It looks as if I need to change it again with a clarification of the review because 5 stars didn't mean then what it means now.

    Those who believe in telekinetics, raise my hand. -- Kurt Vonnegut
  13. G.S. Williams (Member)

    Posted 10 years ago

    I need to go back through this thread and read everything (I'm skimming) but I just want to say:

    In my own reviews, I acknowledge my bias. So far, I've noted that I don't like vampire stories (overdone in the past 100 years)and I don't like diaries, I prefer novels. That's so that readers can know how to look at my reviews -- if they like vampire stories, they can automatically assume a story I rate 3 or 4 stars will in fact probably be a 4 or 5 star story for them, as the vampire thing won't be a problem.

    Part of the idea for WFG (my understanding, anyway) was that by having editors with their own styles, readers can get to know those personalities and find an editor whose style fits their own -- If someone likes fantasy stories, for instance, they'll find that I do too, based on how I review. (that's just an example)

    We need to acknowledge our biases as a way of explaining why we rate the way we do. A 2 or 3 or 5 star story isn't rated arbitrarily, it's rated based on our individual preferences. No one is objective -- the site itself strives for objectivity by letting members and multiple editors review stories, to come up with an average rating.

  14. Miladysa (Member)

    Posted 10 years ago

    @ejames You are correct - I saw a red mist earlier and took it personally - I apologise.

    I do not have the time right now to comment on some of the good points you have made here and for the most part I understand what you are saying and agree with you.

    For the moment I just want to say that from my point of view the thing that irks me the most - and I am rushing and just don't have the time to quote it properly - is something about part of your job being to help the reader to find something worth their time or something like that.

    That's probably the problem with the editor reviews for my part - I want to and am quite capable of judging for myself. I allow the writer to be my guide here. I think - maybe wrongly - that as editors your tastes are probably geared towards more 'literary' fiction than mine or like to give that impression.

    Stuff like Lord of the Flies nearly destroyed my passion for reading and is probably one of the reasons I did not try my hand at writing until now. A few of the editor reviews seem to scoff at the writer for daring to attempt their hand at writing or maybe that is just my perception and I am wrong.

128

Topic Closed

This topic has been closed to new replies.