Brad Wardell's views about technology, politics, religion, world affairs, and all sorts of politically incorrect topics.
Balancing the wants of users and artists
Published on February 21, 2005 By Draginol In OS Customization

Every few months a firestorm wages on WinCustomize.com. The site hosts thousands of pieces of artwork in the form of skins, themes, icons, wallpaper, etc.

And some people can give a star rating to a submission (1 through 5 starS)

On the one hand, often times the skinners (the ones who create the content) have a problem with the whole rating thing. Usually skinners whose works get below 3 stars. Solutions such as eliminating anonymous ratings have been proposed over the years.  Of course, what many would like to see is the elimination of ratings entirely.

And on the other hand, users very much want the ratings. And in fact plenty of users find WinCustomize to be full of "garbage" because we don't moderate very much content. People are free to upload their half-baked skin and unless it's totally broken or really horrendous it's made available. So the casual user finds themselves wading through a lot of stuff they don't like. This hurts the site and the ones who pay for the bulk of the site's costs -- Stardock.

So WinCustomize.com doesn't want to offend skinners but doesn't want to casual users to have to wade through sub-par stuff.

That's where the rating sytsem comes in.  Only users who have contributed to WinCustomize.com (by getting a subscription or having elite access through their works over time) are allowed to rate skins. So most people can't rate skins.  Moreover, the higher the access, the more the rating is weighted.  So if an admin rates a skin, it is weighed far more heavily than a level 1 user who has a subscription.  This way, those who have spent a lot of time doing this and have seen thousands of skins and themes and icons and wallpapers can make a relatively objective rating. 

The second part of the rating system is that those who are not logged on won't see skins,themes, etc. that have 2 stars or fewer ratings. If they log on, they can see it all. Eventually we hope to have per-library rating filters that users can set.

It's a tough thing. The desire to keep everyone happy is an impossible task. Many people who get 2 star ratings or 3 star ratings are simply unaware that perhaps they need to work on their skin. I regularly will go through the galleries and if I see a skin or theme with a rating that seems way too low (or way too high) I'll check the ratings to see if it's being manipulated. It happens. But the reality is, the average skin is a 2.5 star skin which means most skins hover around 2.5 to 3 stars (including my own stuff).

At the end of the day, the skinning community has to realize that it's a symbiotic relationship.  We all need each other.  We need the users AND the skinners. And we can't placate one without alienating the other. So we have to reach a compromise. The rating system is that compromise.  The alternatives -- not allow anything but the very best on would tick off skinners. Or let everything be on with no ratings would make the site unwieldly to use.


Comments (Page 2)
3 Pages1 2 3 
on Feb 22, 2005
I saw that an admin had given your N8T skin a very low rating a couple days ago. It was way out of line with the other ratings it had received, so I deleted that one.

Next I looked at Batman. It was indeed approved with a 3, but not by the same admin. (rating of 3/10, not 3 stars)



So... we have admins abusing the system!
An awful powerful sword to be wielding me thinks. Especially since their ratings carry more weight.
Just dandy...
on Feb 22, 2005

Depends on the skin. A low rating doesn't mean absuse. Let's not let ego take control here.  2.5 stars is the average rating.

IF an Admin is abusing the ratings, the moderator should have contacted me so that we can deal with it.

on Feb 22, 2005
IF an Admin is abusing the ratings, the moderator should have contacted me so that we can deal with it.


Been there, done that.
on Feb 22, 2005

Well I don't recall getting an email on the subject about a particular moderator. I would want to see who this was and then investigate as to why that was the case.

Take your N8T skin for instance. I am currently am the one who rated it lowest (3 stars). I don't dislike the skin, I find it okay. So I rated it as such. Just not my type of skin. Other skins of yours I've rated higher.  On the other hand, there are those who rated it 5 stars. Is my 3 star rating an "abuse"? I don't think so.  I would call a 1 star rating to be abusive though.

on Feb 22, 2005
this system is very bad and really unjust for many skinner which sees
their work destroy by very bad rathing, personally I do not use it any
more because I do not like the injustices !!!
and off course i not say bla!!!bla!!!bla!!!bla!!!bla!!!bla!!!
it's so long !!!
on Feb 22, 2005
Just drop it. I am not looking to tar and feather anyone, nor fight things that are above the law. I have traded emails with you personally Frogboy and when I didn't get any answers or apologies, I chose to let it go. This thread just caused an old scar to itch. You opened the door , I entered.

About my N8T skin. I can handle that no problem. I am glad you had the courage to stand behind your rating. Yes, a 1 rating on almost all wb skins would be a shame. I don't see a 3 star rating from you as anything bad. I would think you have been around the block more than anyone else. If you think a skin is a 2 star or 3 star skin, it most certainly probably is. Maybe I can at least help you to see that showing what and who rates things can possibly do more good than bad. I would be so happy to know what my mentors really think of my work. It gives me something solid to work off of.


Poor Xav73, I think you have had it the worst. Hopefully things will change. Keep skinning!

on Feb 22, 2005
Most of the people who skin for the fun of it, I suspect, are more interested in constructive feedback than in ratings per se. Especially for those of us who do few skins for publication. I would hope even the higher volume skinners would mostly ignore the ratings but if they are important to some people, at least the ground rules and methods are known & discussed. A system that would satisfy everyone and be spam-proof is probably more trouble than it is worth, if not impossible, but I agree that it would be rude, to say the least, to rate a skin without downloading and actually using it. WC can determine if you've downloaded a skin but can't tell if you've actually loaded up & used the skin, so it would be false re-assurance to use the "can-rate-only-if-downloaded" approach. Any rating system will depend on the integrity of the raters and we'll just have to trust them to be fair.

Cheers,
Daiwa
on Feb 22, 2005
Question: How does one rate something as a 3/10 on a 5 star scale? Do admins have the ability to rate on the 1/10 scale then? 3 stars = 6 on the 1/10 scale, yet NT said that it had a 3 on the 1/10 scale.
on Feb 22, 2005
Xav is a great skinner, I enjoy almost all of his work, that is just plain too bad if I understand him correctly and he is leaving..he puts everything he has into all his work and thats pretty much obvious..Ah well.
Just a note of sorrow at the loss..

Zero.
on Feb 22, 2005

Phoon. I rated it as 3 stars. There is no more 10 scale anymore.  It's a 5 star scale.

I don't agree that people should have to have their names show on who rated. That woudl only encourage retaliatory ratings and rating trading and all kinds of other nonsense. Not to mention it would increase the already significant problem of rating inflation that was present before we instituted weighted ratings.

on Feb 22, 2005
Frogboy,

Thanks for the clarification and the thread.

I have just recently started skinning, at least a couple walls and some icons to start, and I know my first reaction is "Oh good, my work made it through moderation" and then I take a look at the rating to gauge how much work I need to put forth to improve.

My stuff is all 2.5 or 3 stars with I think one set of icons at 3.5 stars. For me there will always be a need to improve, even were I to receive a 5 star rating on something - because as we all know, technology keeps on advancing and some things that receive 5 stars today may be considered a 4 star piece in 5 years.

I was thinking that the authors (and I loosely place myself in this category) could place a request in our submission text for comments from those who rate the skin, to obtain some valuable feedback. It seems a few keystrokes would alleviate some of the issues that might be raised by forcing people to comment, and would still allow those who don't feel a need to comment the freedom to not do so.

I personally, would value comments from experienced skinners if they felt it would be beneficial to the advancement of my skills.

Thanks again.
on Feb 22, 2005
Brad,
What about a comprimise...
Show the ratings, without the names. That could only help police the issue better as the admins cannot spot all occurences.

Example:
Skiname
Rating 1: 2
Rating 2: 3
Rating 3: 1
Total Ratings: 3
Now, I realize this does not take into account the different weighting factors involved
but it would help the user gain a better understanding of what is happening
on Feb 22, 2005
Yes, and I foresee it would lead to plenty of calls for admins to remove "low" grades. They have better things to do with their time.
on Feb 22, 2005
Essencay thinks folks are still talking in the 1-10 scale because that's the rating scale showing on the "My Skin" page...
on Feb 22, 2005
Essencay -

Oh, what a tangled web SD weaves.

Cheers,
Daiwa
3 Pages1 2 3