@Ryan: Good point about Sora's idea. And as for the essay, I'm not saying either way - my only job is to be honest.
@Miladysa: Thank you for your feedback - what you've said will allow us to improve the site, and the user-experience. However, it appears to me that you've taken what I said personally, and I apologize if I've given the impression that I was singling you out (and how could I, considering I enjoy your reviews?).
I struggled with the idea of not finishing a whole work before doing a review, but you have to understand the pressure and the reality of being an editor at WFG. We have to cover and evaluate all new listings to ensure that our users are able to find things they'll like to read; so we work under a deadline and a steady backlog of unreviewed, unrated new works. I've written about the process in Novelr here, and allow me to quote the related portion:
An alternative would be greatly appreciated, but at the moment this is the best we've got. Trust me, we're trying hard.
As for possible bias in the review exchange I personally find that idea insulting. Perhaps our reviews are more positive because we have volunteered to review and do not feel obligated to do so?
Do you suggest, then, that we close an eye to the possibility of bias? I believe, Miladysa, that it's fully within our scope to be worried about the quality of member reviews, and about how we're going to effectively leverage the community to help the a new reader find good things to read. We're constantly trying to overcome and compensate for bias, or for gaming the system. In fact, WFG started in part from an analysis I did on the fallacy of small crowds in Pages Unbound. For instance, when I say that a review is purposely polarized as to counteract an editorial rating, I mean this and this. Look at those and tell me that our concerns are unfounded. Chris built WFG from the ground up trying to prevent fanboys from gaming the system and shooting a sub-par work to the top of the listings. This is a challenge that we've faced on a day-to-day basis, working here. But I believe it is a necessary challenge, if we want online fiction to be widely read.
I agree with you that WFG is reader oriented, not writer oriented (though that issue is currently heavily debated by the editors). But let's set aside that debate for a moment and accept that I agree with you: WFG is reader oriented in the sense that we try our best to help readers find good things, new things to read. This is our editors' focus. Your point about how editors need to look at their ratings if they're dissonant from the member's ratings misses this point - our job is to tell you if it's worth your time. Members, however do not have this obligation, and they review for different reasons. Some review because they know and like the author. Others just want to share what they've found. Still others do it because of a review exchange, or maybe they do it because they're not happy with the scores an editor has given. And is it not logical that when you do a review exchange, you don't want to be too harsh, that there is a psychological factor that prevents you from being too negative with the writer you're exchanging a review with? This is the chum factor that I speak of. Mind, we're supportive of the review exchange, because we think it's healthy. But we do have questions, and we're constantly thinking about the possible downsides, because it's our jobs to do so.
I know your time is precious like the rest of us but you have chosen to be in that position and if you are going to do reviews I think you should read through what is available at the time of that review. Personally I do not think it would be a bad idea if a story with only a few chapters was left to develop before they receive an editor review.
Please, consider your words. I feel uncomfortably taken for granted when you put it like that. I too, have family, and all of us have our daily jobs and our real-world responsibilities. We do this only because we're passionate about the cause of online fiction. As for the part about editors witholding from reviewing new works, we've recently had a discussion about that in the private editor forums, and the very suggestion you've proposed have been in effect for two weeks now.
In my opinion some of the editor reviews can border on nasty - I'm not going to single anyone out because I do not think there is any one person in particular.
Do explain, and give me your thoughts, and concrete examples. The whole point of this thread is to overcome some of the perceptions people have about editorial reviews. We're trying to get to the bottom of this and solve it. Is it our tone? Is it the way we handle new writers? How can we improve? Help us.