Frontpage Standards... thoughts

Talk about the mashup scene or anything you want. Section includes Lloyd Recommends and the Monthly Challenges
User avatar
2 Star User
Posts: 478
Joined: Fri Mar 19, 2010 3:00 pm
Location: NYC

Re: Frontpage Standards... thoughts

Post Thu Sep 27, 2012 10:07 pm

Phew! Like Bruce, I tend to put my energy into the Submit forum so I totally missed this thread.
Happy to hear we're going back to quality. Everyone here knows that I take the art seriously and do my best to guide folks towards making their tracks better. I too was losing a bit of gusto when I saw that there were stinkers getting through.
I love the (!) idea.
I also agree that a monthly top ten is a good marketing tool. Bootie and MashUpYourBootz do it and I think it works.
And in regard to how to get everything heard? I don't know. 30 mods? The fact of life is that we all volunteer here and have RL issues to deal with - and every so often we'll all be off working another shift, or getting our car fixed, or changing diapers or what have you. So the only thing I can think of is to add a line to the posting guidelines that says that we all get busy sometimes and to PM a mod if their track has no comments after 7 days. I know it's more work for the poster, but it's a failsafe nevertheless.
What do you think?

User avatar
1 star User
Posts: 234
Joined: Thu Mar 25, 2010 5:01 am

Re: Frontpage Standards... thoughts

Post Fri Sep 28, 2012 5:50 am

garagara wrote:I don't see the point of scouring the net for other mashups to include on the FP though. What's the incentive to post in the forums then? Can someone just send Lloyd or Paul their web address and ask them to check it out every now and then on the offchance they've released a new tune?


I disagree, I think it's a great way of driving up the quality of the FP... and it might even encourage more people in the scene to post here.
____________
"fluffy kittens"

User avatar
Moderator
Posts: 4452
Joined: Fri Mar 19, 2010 7:21 am
Location: around here somewhere...I think..

Re: Frontpage Standards... thoughts

Post Fri Sep 28, 2012 7:22 am

Thanks for the vote of confidence Lee... that was the original reason for doing it :1smiley:
lloyd

No Flash Installed


User avatar
2 Star User
Posts: 478
Joined: Fri Mar 19, 2010 3:00 pm
Location: NYC

Re: Frontpage Standards... thoughts

Post Fri Sep 28, 2012 7:53 am

I just noticed that you guys put up the new forum section about outside tracks. I think that's the right way to go about doing it - it creates an interactive process for the community and adds some transparency. Image

User avatar
1 star User
Posts: 122
Joined: Wed Mar 05, 2008 12:06 am
Location: The Great White North

Re: Frontpage Standards... thoughts

Post Fri Sep 28, 2012 6:44 pm

With the number of tracks being submitted here, I think it's important to maintain tracks on the front page that meet a certain level of quality. General FP visitors would be expecting good quality tracks and shouldn't have to sift through OOT/OOK, poor levels, over-saturated, etc. tracks to find the good ones.

I think it's important that the selection process and criteria be fair, consistent, and objective. If it's not consistent, then mods and submitters alike will be confused and frustrated, and then it won't seem fair nor objective anymore.

If some mods have difficulty being objective and are swayed by liking/disliking, then at least have other mods look at those tracks to verify the selection/non-selection.

onelove wrote:...
Lloyd- you honestly send mixed signals about what should and should not be FP'd (especially as of late) and it made me personally wary of FP'ing tracks. I think this sentiment is probably shared among other mods as well...

As a mash-up submitter, I'm seeing and experiencing mixed signals as well.

I have seen a particular set of criteria applied to a producer as to why his track should not be FP'd. Yet, that criteria was not applied to other producers' tracks that fell under the same circumstances, but their tracks were FP'd. As a mod, it would be confusing about how to make your next selections if you saw this happening. As a producer submitting a track, one or two of those inconsistent scenarios happening on their own track might result in them refraining from posting/visiting on Mashstix for a long time (or leave permanently).

User avatar
1 star User
Posts: 122
Joined: Wed Mar 05, 2008 12:06 am
Location: The Great White North

Re: Frontpage Standards... thoughts

Post Fri Sep 28, 2012 8:14 pm

Sorry for this second post, but I just read the new thread, New Frontpaging Criteria, and noticed that it only actually describes the frontpaging process, but doesn't actually describe any criteria (other than it being the "very best tracks"). So I thought I'd post my thoughts here instead of muddying up the Frontpaging Criteria thread.

If the tracks need to meet high quality standards, then those standards should be definable, quantifiable, or at least describable, so all mods can follow the same standards. And likewise, all submitters should be able to understand whether their track has a chance of meeting those criteria before they even submit their track.

Otherwise, mods still won't know what criteria/standards need to be met, and there's still going to be confusion, frustration, fear, uncertainty, and doubt by all.

You should be able to describe a set of must-haves (or must-not-haves), should-haves, and nice-to-haves. Not meeting the must-haves (or hitting a must-not-have) is a show-stopper... No FP for the track. The should-haves and nice-to-haves can be based on a score for each criteria (eg. Uniqueness: 7/10). And when it reaches a certain score for 2 mods (let's say 8/10), it's FP'd. Or if the average of 2 or 3 mods is above a certain score, it's FP'd. If a submitter questions or challenges the decision, you can get 1 or 2 more mods to evaluate the track, and the final decision stands with quantitative support.

You can then set a weighted score for the should-haves and nice-to-haves. So someone could do well on the should-haves, but below average on the nice-to-haves, and their track still gets FP'd. Or vice-versa... they score average on the should-haves, but they do fabulous on the nice-to-haves and still get FP'd.

As an example, the should-haves could be weighted as 60% of the total score, and the nice-to-haves be weighted 40% of the total score. These weights could be adjusted if the scoring needs to be tweaked.

If not enough tracks hit the front page, re-evaluate the criteria, and or lower the score by .5 or 1 point. If too many tracks hit the front page, raise the score by .5 or 1 point. Or the weights of the should-haves and nice-to-haves can be adjusted.

This type of criteria will also help those mods who are swayed by the like/dislike a track scenario, and will also help when some mods don't like a particular style of music. It will help them keep things objective and in perspective, but will still allow some scoring for artistic qualities so it's not purely technical.

To proceed with this type of criteria/scoring, I recommend the following:
1. Exactly 3 mods come up with the initial draft.
2. Present the draft to the other mods for feedback or possible changes,.
3. The 3 mods that made the draft should make the final decision on changes if the majority of mods are not in agreement on certain changes. So pick your 3 most objective mods for this task.
4. Select 6 different submitters (no mods) to review the criteria and get feedback from them.
5. Have each of the 6 submitters submit their next track and have all mods evaluate each of the tracks using the new criteria. These submitters will know the criteria and will attempt to meet the criteria, so they should score fairly well.
6. Have each mod review 2-3 other mods' evaluations and provide feedback on them. It is best if this is done blindly, so no one knows whose evaluation they are reviewing.
7. Have the 6 submitters review the evaluations that were done on their own track (again, blindly) and have them provide feedback.
8. Tweak the criteria based on all the feedback.
9. Do a concealed trial run. Select about 10-20 other submitters to evaluate their latest submitted tracks using the new criteria (they don't need to know that you're evaluating their latest tracks). These people don't know the criteria, so you should see realistic results reflecting the current state of affairs (eg. 3 out of 10 people get FP'd).

This may seem like a painful process, but in the end you'll have a very solid, consistent standard. And any new mods joining the FP process will find it really easy to pick up the scoring system and be able to give the thumbs up/down to a track with confidence on the first attempt.

The final criteria can then be posted for mods and submitters to see. This will provide additional info to submitters to assist them in improving their tracks.

User avatar
Moderator
Posts: 4452
Joined: Fri Mar 19, 2010 7:21 am
Location: around here somewhere...I think..

Re: Frontpage Standards... thoughts

Post Fri Sep 28, 2012 8:47 pm

hahaha Wow you have thought this through... but seriously you are joking right, you know that the mods here volunteer their time because they enjoy mashups and just want to give a little back to their community.

What you are proposing is impractical and unworkable and to be frank rather offensive, you want us to review the reviewers, I think again you are forgetting one major point, this is not a place of business I have no desire to hold anual reviews, I get enough feedback from members telling me their thoughts on the mods competency, the only person who was letting the side down over the last few months was me, I was tasked with frontpaging more tracks to do this I had to let standards slip, this was noticed by the member, this thread was posted, we formulated a plan that would return the frontpage to it's rightful status and we have moved on.

The guys who post here, are posting mashups, by their very nature they will never be perfect, the one and only criteria that any of the mods need to worry about is whether they actually enjoyed the submission.
If two mods enjoyed it then that's good enough for me...

Allow me to keep my own counsel when it comes to deciding who I choose to have on my team but if you have any concerns about any team members ability to perform his duties speak to me in private... and I'll be only to happy to listen.
lloyd

No Flash Installed


User avatar
2 Star User
Posts: 478
Joined: Fri Mar 19, 2010 3:00 pm
Location: NYC

Re: Frontpage Standards... thoughts

Post Fri Sep 28, 2012 9:52 pm

Qubic: I just wanted to jump in and say thanks for going through the effort to come up with this complicated system. Two issues occur to me right away:
1) when it comes to artistic expression and trying to determine quality, there's the old adage about quality and porn: you know it when you see it
2) when it comes to mashups in particular - there's a special a-ha moment that comes with the best ones - you know it... when one part of your brain is bopping to a familiar mental and one part is bopping to a familiar pella and some other part clicks and says either that's funny or that's clever or that's magic.

I don't think there's any way to measure any of this. We can measure technique extensively with a system like yours but in the end it will always come down to the magic and there's no way to systemize that.

I think that gathering a bunch of people together who care about the art and care about helping others with the art is a pretty sane approach -- especially when we're all volunteers.

So let's see what happens now that the bar has been raised again - and that the (!) system has been added. I've got a feeling that it's all going to work out. :1smiley:
Yours in mashups,
DG

User avatar
1 star User
Posts: 122
Joined: Wed Mar 05, 2008 12:06 am
Location: The Great White North

Re: Frontpage Standards... thoughts

Post Fri Sep 28, 2012 10:34 pm

DAW-GUN wrote:Qubic: I just wanted to jump in and say thanks for going through the effort to come up with this complicated system. Two issues occur to me right away:
1) when it comes to artistic expression and trying to determine quality, there's the old adage about quality and porn: you know it when you see it
2) when it comes to mashups in particular - there's a special a-ha moment that comes with the best ones - you know it... when one part of your brain is bopping to a familiar mental and one part is bopping to a familiar pella and some other part clicks and says either that's funny or that's clever or that's magic.

I don't think there's any way to measure any of this. We can measure technique extensively with a system like yours but in the end it will always come down to the magic and there's no way to systemize that.

I disagree. I do enough analysis in my job that I've come to realized that even things you think can't be qualified, in fact, can be qualified or rated.

The "A-ha" factor that you talk about usually stems from something termed as expert knowledge. It occurs in every field, and people mystify it by calling it the x-factor, or a-ha moment. The reality is, it is merely a number of different variables that your mind is taking into account, and your mind produces that a-ha moment. You can, in fact, break down those variables and analyze them.

But before we go down the long road of analysis, you can easily accommodate it in the criteria by simply adding an "X-Factor" field in the criteria that the mods rate from 0 to 10, and it has a certain weight to the overall total.

However, in my opinion, no matter how big an X-factor it has, if it doesn't meet the must-haves, it ain't gonna be FP'd.

DAW-GUN wrote:So let's see what happens now that the bar has been raised again - and that the (!) system has been added. I've got a feeling that it's all going to work out. :1smiley:


It's nice to say that the bar has been raised, but the reality is nobody knows where that bar really is in the first place. There's no real criteria or standards defined.

And I've already seen a mash-up denied FP status with the new (!) system based on a set of criteria (reasons) that was not consistently applied to other mash-ups in the same scenario, but they were FP'd. As I mentioned in a previous message, not being consistent with evaluations breeds confusion and frustration. So those people hit with the inconsistent decisions walk away with a big WTF... and no reasonable or tangible explanation why their track has to meet different criteria than someone else's track. A few of those scenarios happening to the same individual, and they'll say "why bother" regardless of how much people care about the art and helping others.

User avatar
3 Star User
Posts: 852
Joined: Sun Nov 27, 2011 5:32 pm
Location: The uncharted backwaters of the unfashionable end of the western spiral arm of the galaxy

Re: Frontpage Standards... thoughts

Post Sat Sep 29, 2012 4:26 am

As accurate and effective as that sounds, Qubic, it would take far too long. As mentioned, the moderators are volunteers, we have a life outside of Stix and if there was reviews, scores, reviews of the reviews and agreement with the majority of the mods, we'd take at least a week to FP one track. Hardly ideal.

With regards to actual criteria, I would like to point out this section in the Users Guide:
What gets your track to the Frontpage is how you execute [a] good Idea. So our team (Team Stix) are tasked with scrutinising your submission listening for substandard audio, off tempo vocals, clashing of key, the click and pops of badly trimmed loops, disparity of vocal levels...
http://www.mashstix.com/forum/viewtopic.php?p=32841

That's pretty much the 'must(/not)-haves'.. as for 'nice-to-haves', it really depends on the producer and track. Sure, a sprinkling of extra original production is nice to have, but if the producer is not able to, I would think it wrong to ask them to. Likewise, if they saw that 'it's nice to have some extra production in your track' but know they're not up to it, that would probably discourage them from posting it and then it'll never get on the FP. Pehaps a clearer set of goals to aim for to get on the FP is in order, I don't know. I'll have a think.

PreviousNext

Return to Stix chat

Who is online

Users browsing this forum: No registered users and 11 guests