Rating Closed »

I think it's time we had a ratings talk

by mangosango on February 21, 2012

EDIT:
Also, try to keep everything to one window. There are quite a few shots on here that have two screenshots combined into one image. I know you may want to show off your desktop wallpaper, but this is a customization website, not a desktop showcase site. Unless you have a very, very, good reason to use more than one desk (you have too many skinned apps to fit on one screen), keep it to one. Thanks =)


OP:

Just thinking out loud. Feel free to discuss in the comments.

I’ve seen lots of 100s here lately. While it’s good that people are finding things they like, I don’t think that handing out perfect scores is in keeping with the spirit of PF. I see PF as a place I come to when I want real critique on my work, not just “yay, it’s great!” praise. A low score may be disheartening, but it can also be very good for you! I think that PF is drifting away from this.

For the most part, admin/mod scores are fairly well balanced, with some admins & mods being more generous or critical than others. I try to offer critique with each score, and generally I think the other admins/mods do a decent job of that as well.

In any case, here are some guidelines that I propose – for the benefit of raters, and for authors, who might be insulted by what they feel is an unfair rating.

Ratings (for screenshots):

  • 100 – Perfect. Flawless. Could not be better in any conceivable way. imho, 100s should only be given when you cannot think of any way to improve the submission. These are are shots that push the boundaries and go on to really impact customization in a big way (like pep’s snohetta, for instance :3).

  • 95-99 – Excellent. Extremely good. Submissions that are amazing, and push the boundaries, but suffer from 1 or 2 minor drawbacks, like padding issues or minor font discrepancies. These are god-tier submissions.

  • 90-94 – Great. Very, very good. Submissions that are almost there. These shots bring a lot of new ideas to the table, or demonstrate a new twist using a pre-existing style. They may suffer from 1 or 2 minor drawbacks, or have one semi-major flaw (maybe the icons just don’t fit).

  • 85-89 – Very Good. Submissions that are well-executed and composed. Don’t necessarily bring new ideas to the table or push the boundaries, but nonetheless show a good mastery of customization. An explorer window must be shown.

  • 80-84 – Good. Submissions that, apart from a major flaw, or 1 or 2 minor flaws, are good. These shots demonstrate a good understanding of customization, but have lots of room for improvement. An explorer window may or may not be shown.

  • 75-79 – Above average. Submissions that show a decent understanding of customization, but suffer from 1-2 major flaws that prevent it from being a good shot (maybe the style doesn’t work, or the shot has too many elements that don’t fit together).An explorer window may or may not be shown.

  • 70-74 – Average. A decent shot. Not good, not terrible, just average. There are some good elements to the shot, but they aren’t utilized very well, and don’t contribute to the whole. I’d like to stress that these aren’t terrible shots, just average.An explorer window may or may not be shown.

  • 60-69 – Below average. Questionable. These shots demonstrate a superficial understanding of customization, or a lack of effort. Feels thrown together. An explorer window may or may not be shown.

  • 50 and below – Good chance for moderation. An explorer window is not shown. Large parts of the setup are not customized, and the shot demonstrates an apparent lack of customization knowledge.

So there you go. Thoughts? Discuss below.


mango

User Comments

Avatar

dracofusco wrote:

Great post mango, you definitely have a point. Pixelfuckers, at least from my understanding and experiences, is a site where you come to learn how to really customize your computer, and how to learn the artform of screenshots. Handing out perfect scores isn’t really fair to the author of the shot because it makes them feel like they have nothing to improve, when in reality, almost all of us here have something we can improve in our shots. I’d also add a shot that gets below a 35-40 should be moderated, I feel like that is too poor quality of a shot to be allowed. Again, nice post, definitely agree here.

Pfphoto

TheBoxTheory wrote:

Couldn’t agree more! PF actually helped me see my mistakes and get a bit better! [Also I got a bit more active and started creating rather downloading and placing which taught me a lot ]

Although I have run out of ideas,atm [It is just me or the community seems to have too ?] ;)

Great post btw! :P

Avatar_pf

neiio wrote:

Awesome post! You obviously put a lot of effort into your ratings scheme and I couldn’t agree more. I just hope everyone gets a chance to read and study and also start applying it.

Great job mate.

Nanu_new

Nanu wrote:

Good issue. I sometimes give out a higher rating then needed but will know in future.

Avatar

akka wrote:

I haven’t been around much lately, but I much appreciate you hankering down on things, Sang.

I like your ratings list, and appreciate Average being 70-74. People slap an 85 on those too often.

Avatar

akka wrote:

Hunkering, rather.

Messingaround

Man.I wrote:

I’m with you there Sang. This is a community of gasp skilled designers from a variety of communities ( mainly Deviantart and Customize.org).

PF should be curated like any good art exhibit would be — with reasonable scrutiny, and care.

Hackergotchipf

crshd wrote:

I’d give this post a 100.

Untitled-1

4nt1p0p wrote:

I like these guidelines what screenshots counts. But I hope that the fact that I dont show explorer window while submitting rainmeter + cad suite will not give me less rating, since I wouldnt see a point.

Who-are-you%20(200)

phaxn wrote:

I agree with your idea! But rather than having a 0 – 100 rating with an explanation where most of the numbers mean the same “85-89 – Very Good.”, I’m thinking it might be better to change the ratings-system to be more self-explanatory, so people won’t have to

Example:
Rate:
Please be critical when you rate this!
- Flawless
- Excellent
- Great
- Good
- Average
- Needs more work
Explanation of ratings

With an “explanation link” at the bottom where people can read an explanation to each rating (in exactly the same way you did it in your post above!).

I’ve been using PF for a long time as an inspirational source and have always been confused over the 0 to 100 rating system. From my point of view comments are often much more valuable and I think swapping the rating-system for an “appreciation button” would be the best solution, directing people who just want to “like” the work to the appreciation button and directing the users wanting to give some proper critic to the comment-section

Hope some of these ideas can be of use! :)

Log in or register to post comments!