Helpfulness of Ratings

The latest Scientific American contains an article that might be of interest: how well do Internet ratings systems work?


http://www.scientificamerican.com/article.cfm?id=manipulation-of-the-crowd


--Shelley


This supports the theory of a friend working in customer support at a fastfood call centre. They only get lovers and haters, lots of haters, but overall a tiny fraction of customers. Her theory is silence suggests mediocrity.


But it could as easily suggest contentment, or laziness. So silence is better than abuse, but says little else. It does make me wonder how many readers will return to drop tips in a jar, if such a small percentage bother to click a star rating.


I can't find the ref now but I read somewhere digital music sales were being manipulated to produce top 50 hits by bulk self purchasing. Damn lies and statistics. Nothing changes.


Isn't this why youtube got rid of their star ratings and changed it to thumbs up/down?


I've been experimenting with something similar on my blog posts (replaced stars with thumbs) -- but haven't noticed much difference in the number of ratings given.


I get lots of love and the occasional virulent hate, so hopefully that means I'm not mediocre? ;)


The ratings on the Sims 2 story exchange drove me crazy. My first Sims story was the highest rated for a while, and then the one star ratings came. And I knew it wasn't objective and it didn't reflect the quality of my story at all, but I didn't know who was doing it or why and who knows, maybe they did have a valid reason.


So I stopped doing stories until Sims 3. The Sims 3 exchange doesn't have ratings, but it is horrible. So everyone's using blogs instead, and you can bet that I did not enable ratings on my blog. And so, with only real comments to go by, I felt happier and more motivated and actually finished a Sims story for the first time ever.


And the haters had to resort to anonymous secrets in the Sims Secrets community and so I could see that their opinions were not valid. And kind of scary and obsessive.


But yeah - after my experiences with star ratings, there's no way I would accept them as an objective marker of a product's worth.


I thought everyone knew internet ratings were bollocks. That was originally the point behind WFG's editor reviews. A cogently written review, be it positive or negative, is the only way to gauge the quality of any work. A few words or JPEGs of apoplectic love or hate doesn't say a goddamn thing.


That's why I'll always be pleased that I've been publishing and selling work for near three years and have yet to get a solidly negative review.


Regards,

Ryan


There are two separate problems associated with online ratings; the extreme scores discussed in the Scientific American article, and attempts to "game" the system. I remember an instance years ago in which the "gaming" got completely out of hand; this was on a fiction directory that linked to my site. At first all went well, and then someone apparently became envious of the better-ranked sites (mine included) and sent out a tsunami of "one" ratings (it was a one-to-ten scale) to wipe out everybody else's score. After a few rounds of this somebody else decided to retaliate and plastered the directory (not just the miscreant) with more "one" ratings. Pretty soon everyone's score was too low to ever rise again (there were so many votes that honest new "ten" ratings couldn't make a dent) and eventually the directory folded.


Web Fiction Guide seems pretty well protected against that kind of thing.


--Shelley


True Shelley, we luckily don't seem to get that nasty stuff and I think there are some safeguards and monitoring built into the system, and it also helps that we have a great core group of authors/readers/reviewers who keep up a very professional and positive standard. Yay you guys :-)


We also don't seem to get the extreme ratings. I dont' know the stats, but it looks like the vast majority of ratings are between 3.5 and 4.5. This is probably realistic, with a bias toward the positive because people tend to read things that at least catch their attention initially as being good.


I think the major problem here is homogeneity in the population of editors, founding members, and regular reviewers in terms of prefered genre. For better or worse WFG specializes in fantasy and tales of the supernatural. This makes WFG a great site to find fine examples of that genre, but makes it difficult for submissions in other genres to get the attention they may deserve. I only hope that in the future a broader spectrum of readers will find us...it will also help if people who like to review (myself included) make a point of sometimes thinking outside the box and trying something different from what we would normally read.


Hi Fiona,


Your last paragraph basically sums up my frustration with the webfiction world. There is a major focus on sci-fi/fantasy fiction these days, and that's awesome. Obviously, there is quite a large fanbase for it and it's a very big idea right now. But for those that don't write it, you begin to feel like an outsider. I'm sure it's not intentional (no one has ever been intentionally hurtful, petty or mean to me on this or any other webfiction community), but you begin to feel like your story is not worth crap if it doesn't have a fairy in it. Or if it's not set in a university.


It's disheartening at times, but I hope (now that I have more free time) to weed out some stories that are not fantasy-based and give them reviews, so others can see that there is more out there than they might realize.


I wouldn't even say there's a major sci-fi bias; fantasy works outnumber everything else on the WFG put together by about 2:1. If you look at the 'popular' section, you'll find only a handful of SF works (or any non-fantasy works) in the first few pages.


I guess a lot of people just like their escapism as escapey as possible.


Regards,

Ryan


Probably good to keep in mind that the internet appeals to escapism and the technically minded--the former more than the latter--hence the skewed numbers as far as internet fiction goes. I'm sure as the internet becomes a staple of our every day lives, we'll see a drastic shift in the numbers--and the ratio of fantasy fiction versus not-so-fantasy fiction will start to normalize itself.


S'funny, back when I first wrote a serial in '00, every community I tried to integrate with was primarily made up of soap opera-style writers and series, and I was the one feeling like an outsider!


Oh--assuming this is what you're talking about--I'd forgotten about the enormous fan-fiction networks out there (which tend to be very soap operaish, and are probably as old as the internet itself)--although I think the majority of them *tend* to fall within the terms of fantasy.


The balance here is very different from what you see in the bookstores and libraries; yes, there is a strong bias towards fantasy on WFG. But the books in the bookstores (and libraries) are selected because they are the type people want to read, whereas the stories here are those that the people here want to write.


Which even in a group of writers (and I'm told that not everyone who comes here writes) is not necessarily the same thing. At a local writers' event I once met a young man who told me that he loved Jane Austen. I tried to interest him in my critique group but was told that though he loved to read Jane Austen he only liked to write fantasy.


Makes an interesting question, actually: do you write the same sort(s) of fiction you read? And if not, why?


Dary wrote: S'funny, back when I first wrote a serial in '00, every community I tried to integrate with was primarily made up of soap opera-style writers and series, and I was the one feeling like an outsider!


Way back when, somebody realized that serials would keep readers coming back for more . . . and for the post-Dickens media "serial" basically meant "soap opera." (Actually, maybe I shouldn't exclude Dickens from this.) So the slant in the Web's early days was in that direction. The first Web serial ever, The Spot ( http://en.wikipedia.org/wiki/The_Spot ), was something like an early "reality" program (of a soapish variety) and though it went bankrupt a couple of times it was an enormous hit, considering the amount of overall Web traffic back then.


--Shelley


*sighs*


Just to be clear "The Spot" was *not* exactly the first Web Serial ever, just the first to really get noticed. People have been self-pubbing their fiction in serialised form on the the internet since before there was a web, I fully expect the first web serials appeared around the time Mosaic appeared. But no one called them that then. You put a bunch of people near a many to many medium and some of them will put fiction on it...


Interesting bit of history - I'd never heard of the Spot. To be honest, doesn't sound like something I'd mind missing. There's enough of that sort of thing on TV. Now what's lacking on TV, interestingly, is good adult fantasy (with a few exceptions eg. Buffy, the X-Files), but certainly not much alternate-world fantasy beyond children's cartoons. So maybe it balances out.


Shutsumon wrote: People have been self-pubbing their fiction in serialised form on the the internet since before there was a web, I fully expect the first web serials appeared around the time Mosaic appeared.


Computer-distributed fiction does go back a ways (my serialized adventure began in 1991, on a BBS). My post was in response to Dary's comments about Web soap operas, which for a while were indeed rather dominant. I brought up The Spot because it got the soap thing going. There may have been some confusion because I paraphrased Wikipedia, whose article begins, "The Spot, or thespot.com, was the first episodic fiction website (1995-1997) . . ."


Back around the time The Spot began the big thing was online literary magazines, which were patterned (naturally) on their printed forebears, with several different contributors each. Then soaps became a big thing too.


--Shelley