PDA

View Full Version : Mandatory Comment Cards for a score below a 6


Sledneck
08-10-2010, 08:55 AM
Should there be a mandatory comment card for a score below a 6? Has this ever been discussed/voted on? Does anybody else support this? If enough members are interested in making this a rule how do we get it on the table?
__________________

Cue's Your Daddy
08-10-2010, 09:00 AM
we actually recieved a 5 and were given a comment card this past weekend. So a bbq judge doing the right thing

Ford
08-10-2010, 09:02 AM
No. Most comments are just not meaningful. Especially for appearance. I didn't think it looked average is not going to help you. For taste maybe but if you get one low score and the rest are 7-9 and the one low says too salty how does that help you.

Just my opinion but the times I did get comments it just pissed me off.

biggs98
08-10-2010, 09:07 AM
I would love comment cards good or bad. I think it would be great to see what judges say about your food. Especially when you get 8's and 9's from 5 judges and 1 gives you 6's. I would like to know the discrepency. Maybe the got a piece of fat or a bone. I know I'm in the minority on this. It's just something I would like to see.:pray:

Sledneck
08-10-2010, 09:13 AM
Are comment cards discussed /encouraged during a judges meeting? Maybe a few suggestions from the reps as to what cooks are looking for in those cards? How about a comment card with a list of standard check off boxes and a few open lines at the bottom for other?

Buster Dog BBQ
08-10-2010, 09:20 AM
I don't want to hijack your thread, but at the same time I would like to see a minimum score a non CBJ can give.

Isn't it the job of the table captain to see the scores of the judges? And if they do see a swing of 3 points,potentially pull the low judge aside and ask why? Or maybe that is the reps?

I would like to know why anything is ranked 5 or below. That way it also prevents errors in entry like a 4 intread of a 9 or something if a card is required for low score.

Of course on the other end, judges may not want to do that and never score below a 6.

Slamdunkpro
08-10-2010, 09:28 AM
Just because you didn't get a comment card doesn't mean the judge didn't write one. The KCBS reps edit/discard them at their discretion.

Alexa RnQ
08-10-2010, 09:34 AM
I absolutely think that if a judge really wants to give out a 5 or below, a one-word descriptor (i.e. salty, spicy, fatty, dry, tough, raw, whatever) isn't asking for too much to justify it.

The value is not in whether a competitor will turn himself inside out over one 5; it's in making that judge think for one additional second about a snap decision that could too easily tank a score. It's a reminder in process that 6 is average, and if a sample is truly below average they'd have a good notion why.

DawgPhan
08-10-2010, 09:37 AM
In 4 years of contests I have never gotten a comment card. The judges say everything they need to say with the number, but I do like the idea of making them think about what they are scoring.

though I think that with improved training and repetition we could eliminate the out of whack judges.

Alexa RnQ
08-10-2010, 09:42 AM
I'm also very much in favor of preprinted scoring cards where boxes are checked, as described above.

ZILLA
08-10-2010, 09:47 AM
How about everyone gets 8s and 9s then a rubber stamp of a big smiley face on your comment card or maybe a gold star. You want higher scores? Learn to cook better. A low score should be all the comment you need. If you're a cooker you should be able to figure it out like all the others that took the time and made the effort and investment to get it right without some self righteous CBJ telling you what direction to head. You want a CBJ to tell you what's wrong? Really? You would end up with rubber stamp BBQ.

Jorge
08-10-2010, 09:52 AM
How about everyone gets 8s and 9s then a rubber stamp of a big smiley face on your comment card or maybe a gold star. You want higher scores? Learn to cook better. A low score should be all the comment you need. If you're a cooker you should be able to figure it out like all the others that took the time and made the effort and investment to get it right without some self righteous CBJ telling you what direction to head. You want a CBJ to tell you what's wrong? Really? You would end up with rubber stamp BBQ.

We should all have a system where you don't have a clue if you don't make final table?:-P:becky:

Rub
08-10-2010, 09:55 AM
FBA judges are told to write a comment on any score of a 7 or less.
We score in half point increments to 10 as a high.

Harbormaster
08-10-2010, 10:04 AM
At my first comp I received one comment card.
It said the skin on my chicken thighs was perfectly cooked.
We turned the thighs in skinless.
Just sayin'.
YMMV.

ZILLA
08-10-2010, 10:21 AM
We should all have a system where you don't have a clue if you don't make final table?:-P:becky:

Sure you have a clue. Not hitting final table says your food sucks! You need to practice! All the other cooks have better food. That's all the clue you need. Perhaps you need to ask another BBQ man for advice. Maybe listen to your wife when she gives you advice. So my score card tells me I'm 30th out of 35 teams. The comments basically say my food didn't taste good. Ummm I already knew that! I need a CBJ for that? A score card with comments from an individual who can't cook yet is a CBJ? OK....I didn't hit final table? Gee I guess I need evaluate my own food. Maybe I need to learn some better cooking techniques.

Maybe there should be a CBJ cooking school for the cookers. That way the CBJ can teach you how to cook BBQ they like! HA!

KC_Bobby
08-10-2010, 10:28 AM
I like that comment card idea, but I think inconsistent scoring has more to due with non/new CBJs scoring differently. I'd like to see KCBS track past judges scores and then assign them to tables so the average CBJ score is as close as possible, then divide up the new and non-CBJs equally among the tables (I know they already try to do that later part).

I did a comp this past weekend where one team's brisket score was:
32.5714 34.2858 34.8572 30.8572 21.7142 32.0000
Which one doesn't belong? Did they get an 855 or a 656 along with all those other 8s and 9s? Hope the table capt said something ... or maybe the entry just didn't have near enough salt in it's taste profile for that judge (that's sarcasm referring to a post from a different forum). At least that was only one rough judge.

Slamdunkpro
08-10-2010, 10:31 AM
How about everyone gets 8s and 9s then a rubber stamp of a big smiley face on your comment card or maybe a gold star.
More hijacking

Score "creep" is a big issue - 7's used to be good, 8's great and 9's rare. Now it's almost expected to be given 8's just for making it to the turn in table without tripping.

Jorge
08-10-2010, 10:33 AM
More hijacking

Score "creep" is a big issue - 7's used to be good, 8's great and 9's rare. Now it's almost expected to be given 8's

Actually, 9's used to be common and 180's weren't as rare. There were some changes to judging instructions and scores came down. There was another change, and scores are allegedly on the rise.

Sweet Breathe BBQ
08-10-2010, 10:55 AM
A while back we received two 4's from a judge, while the rest of the scores ranged from 6-8. I think that should warrant a comment card. I think that this ties back to the fact that often times there is a wide range in scores. It is really hard to make improvements when there is such a division amongst the judges themselves.

Balls Casten
08-10-2010, 10:59 AM
I dont want a card from the ONE guy who didnt like my flavor. He's going to score the way he is going to score. And I'm not going to make changes because of his score. Its the table captain's responsability to be sure he is scoring according to KCBS rules not mine.

However with my great cooking skills, I would like to see cards when I toss 141 on the board. :-)

QueTPiesBBQ
08-10-2010, 11:01 AM
I got a comment card on a 5 one time that said it tasted like ketchup. Go figure.

ModelMaker
08-10-2010, 11:12 AM
Just because you didn't get a comment card doesn't mean the judge didn't write one. The KCBS reps edit/discard them at their discretion.


Absolutley untrue.... Who told you that this is the case or are you guessing that's what happens?
During judges instructions we are told that ALL cards get attached to your score sheets, period.

As far as mandatory comment cards at any given number, whatever one you choose (4,5,6) it puts a mental block that says if I give that number I have to fill out a card. Chances are you'll never see that number or lower again. Is that how you think judging should go?
Remember a 5 is just simply below average, not crap, not inedible, 5 is not that far out of whack. If your food is just slightly less than average your gonna get a 5. I have given out several 5's this year.
And as a rule I have filled out 4 comment cards also. They simply said what it was that made that sample a 5. Thats how the system works.
Tis true all judges don't always get it, but find anything in life that is just the way everbody wants it.
Ed

ModelMaker
08-10-2010, 11:16 AM
I got a comment card on a 5 one time that said it tasted like ketchup. Go figure.


So the judge didn't like your sauce and told you why.....
what more do you want?
Ed

rweller
08-10-2010, 11:24 AM
I have judged or Table Captained 10 contest this year and I beleive that every REP while doing his speach has told us if we score a 5 or below to please write a comment card as to why you are giving that score. That way they have a justification to the team and it will also let the team know why it was given.
As someone mentioned above I'm sure there are some judges that won't give a 5 or below because they have to fill out the comment card.
But then you have some judges(not many) that give them out left and right and it doesn't bother them to fill out the cards.
When I table Captain I do look at the judges cards for low scores and if I see one judge giving a lot of them or more than the rest of the judges I do let the REP know. It is the REPS jobs to talk to them not the TC.

Slamdunkpro
08-10-2010, 11:35 AM
Absolutley untrue.... Who told you that this is the case or are you guessing that's what happens?
During judges instructions we are told that ALL cards get attached to your score sheets, period.
Ever worked an event? If not I suggest you do before you make statements like this. I've witnessed various reps discarding comment cards on multiple occasions.

Alexa RnQ
08-10-2010, 11:56 AM
You want higher scores? Learn to cook better. A low score should be all the comment you need. If you're a cooker you should be able to figure it out like all the others that took the time and made the effort and investment to get it right without some self righteous CBJ telling you what direction to head. You want a CBJ to tell you what's wrong? Really? You would end up with rubber stamp BBQ.
I think our record would substantiate that we have indeed learned to cook better. And we still see way too many rogue scores to just blow it off. If one is fortunate to live in an area with a plethora of experienced and consistent judges, it's a non-issue for you -- that doesn't mean it isn't an issue for others.


As far as mandatory comment cards at any given number, whatever one you choose (4,5,6) it puts a mental block that says if I give that number I have to fill out a card. Chances are you'll never see that number or lower again.
Every judge that fills out a comment card now for those numbers refutes that idea. I would not, however, advocate a mandatory card for a 6.

dmprantz
08-10-2010, 12:07 PM
How about everyone gets 8s and 9s then a rubber stamp of a big smiley face on your comment card or maybe a gold star.

I apologize for the threadjack, but why is it that every time some one asks for ways to improve consistancy in KCBS scoring, some one has to come on the thread and sarcastically suggest that every one get a 180 and a trophy like a children's karate tournament? Is it always the same person or two? I don't think that any one here is in doubt of the fact that BBQ comps are in fact competitions and that some people do better than others. I also don't think that any one doubts that there are some teams, here in The Brethren and otherwise, who are just better than than other teams, in product, consitency, or both. Still, people get frustrated with left field scores, and it comes up enough that maybe, just maybe it's worth having a conversation about it.

Some competitors are just so darned good that they never get anything below an 8, great for them! Other's have resigned themselves to the fact that the stray 6 or 5 or whatever is going to find its way into their scorecard no matter how good they feel their product was. That's very admirable. Others still want to be good, want to improve, and want to ensure that when they do get a good product out there, they get rewarded for it. I'd like to suggest that those who know that their product is always great and never get a bad score discuss that amongst themselves. Is it really productive to imply that people are asking for a GC trophy every time scoring anomolies are discussed? Does that ever really help anything? Am I the only one who feels this way (Very likely, so don't worry).

Again, sorry for the threadjack. If mods feel this should be its own thread, lemme know and I'll copy and paste it there.

dmp

ZILLA
08-10-2010, 12:14 PM
More hijacking

Score "creep" is a big issue - 7's used to be good, 8's great and 9's rare. Now it's almost expected to be given 8's just for making it to the turn in table without tripping.

Hijacking my arse! You need to read my post and it's intent "Slamdunkpro".

Let me carify....Quit whining! Is that a little more clear? Go cook and live with it! How about this one...Not everybody is going to like your food or thinks as much of it as you do.

ModelMaker
08-10-2010, 12:20 PM
Ever worked an event? If not I suggest you do before you make statements like this. I've witnessed various reps discarding comment cards on multiple occasions.


No I have never stood at the score input table but the statement I made
" ALL cards get attached to your score sheets" is a direct quote from the president of the KCBS if your reps are " discarding comment cards on multiple occasions" they appear to be acting out of compliance with what Mr Lake is instructing.
Perhaps you should follow up with a communication to Mike as this only defeats the effort to help cook teams understand their scoreing questions.
Ed

Bentley
08-10-2010, 03:03 PM
Should there be a mandatory comment card for a score below a 6? Has this ever been discussed/voted on? Does anybody else support this? If enough members are interested in making this a rule how do we get it on the table?


I would like to see it mandatory, but don't think it ever will be. I believe KCBS might be worried you would drive those judges away if they feel pressured to do so...I think that would be the best thing that could happen to the judging pool...You need judges who are as commited to excellence in the judging as you do a cook competing. If the judge is not willing to make that commitment, I hope they will be driven out. I know as a competitor I have a little different view of judging then someone who does not. It seems we are constantly adding new judges here in CA. It probably took me a little less time to take the personal aspect of judging out of the equation because I compete, but it simpily does take time...I dont know how many times I have sat at a table with a non-CBJ or one that has judged 1-2 contest and they will give a score below a six and not even consider advising the team why...And I agree, the comment needs to be constructive and informative, if you cant articulate why you gave it a 3, 4 or 5, you have no business sitting in the seat.

I wish CBJ classes would concentrate on what makes BBQ above or below average, not is there Red leaf or kale in a box, how often does that happen at a contest?

But if I give a judge in a class a choice of Brisket, Butt, Ribs or Chicken, each one cooked to 160, 195 and 215 do any of you competition cooks think there may be a difference in the tenderness of each of those...If I submit a non seasoned, a "properly" seasoned and a super coated with rub and sauce, do you think there maybe a different flavor profile...never understood why things like this are not done at classes. You know I am not gonna be sitting at the Algonquin Round table anytime soon, if I can think up this stuff, I am sure there are a lot smarter folks at KCBS that should be able to too!

Sledneck
08-10-2010, 03:24 PM
FBA judges are told to write a comment on any score of a 7 or less.
We score in half point increments to 10 as a high.

If only an organizer up here would have the balls to sanction an FBA contest up here in the northeast, sigh...........

Sledneck
08-10-2010, 03:33 PM
How about everyone gets 8s and 9s then a rubber stamp of a big smiley face on your comment card or maybe a gold star. You want higher scores? Learn to cook better. A low score should be all the comment you need. If you're a cooker you should be able to figure it out like all the others that took the time and made the effort and investment to get it right without some self righteous CBJ telling you what direction to head. You want a CBJ to tell you what's wrong? Really? You would end up with rubber stamp BBQ.
I retyped a response to this a few times and decided it was a waste of time, move on buddy

Candy Sue
08-10-2010, 03:47 PM
(This is only my opinion!). I think the reasoning behind comment cards is good, but the mandatory card if you give a 6 or below has one effect -- it makes judges score higher. It's the lazy factor, give a bad entry a 7 and you don't have to do anything. Look at the number of 180s this year compared to 2009 or 2008 even. After the rules were changed to start at 6 and go up or down, judges used the whole spectrum of numbers and there were very few perfect scores. I can remember back in 2004 when I attended the banquet, there were so many 180s that it really didn't mean much. I would hate for 180s to get back to that point because we've essentially put in place a 7-8-9 scoring preference. (Did I say that this is only my opinion??? Ir is and I hate getting comment cards on my entries!)

Slamdunkpro
08-10-2010, 04:00 PM
Hijacking my arse! You need to read my post and it's intent "Slamdunkpro".
Um,,,the "more Hijacking" was in reference to my comment, not your post


Lighten up Francis.

Scottie
08-10-2010, 04:02 PM
I agree about the mandatory cards effecting judges and them scoring up... i do know when I get the cards, all it does it upset me. The best comment is the old "tastes like lighter fluid"... Um, OK, but I am a pellethead. (Using the best pellets made out on the circuit!!!) :becky:

On the other subject. I don't have any data, I am sure you do on this subject. But are there more 180's now, compared to when the new rules went in to effect? How have they increased over the years since the change? I am also finding more and more ties at the contests that I am cooking. Is that a coincidence? I have no clue. Is the system perfect? No. but I am not sure if there is a perfect system...


(This is only my opinion!). I think the reasoning behind comment cards is good, but the mandatory card if you give a 6 or below has one effect -- it makes judges score higher. It's the lazy factor, give a bad entry a 7 and you don't have to do anything. Look at the number of 180s this year compared to 2009 or 2008 even. After the rules were changed to start at 6 and go up or down, judges used the whole spectrum of numbers and there were very few perfect scores. I can remember back in 2004 when I attended the banquet, there were so many 180s that it really didn't mean much. I would hate for 180s to get back to that point because we've essentially put in place a 7-8-9 scoring preference. (Did I say that this is only my opinion??? Ir is and I hate getting comment cards on my entries!)

CaptTable
08-10-2010, 04:08 PM
I'll tell you right up front that yes I do cull comment cards. Many times there are comment cards filled out improperly such as no team number or a comment, but no scores written down to go along with the comment. AND, if the comment is not CONSTRUCTIVE, it may get tossed. I've tossed one that said a team's meat "smelled like vomit". I've tossed one that said "it sucked". What good would it have done to pass those on to the teams. NONE. I only wish I could have determined the judges that wrote them. But, I did not read them until later.

Whenever I am the Rep conducting the judges' meeting, I always stress that comment cards must be constructive and, also, remind them that they are voluntary. However, I will ask them to fill one out or, at least, tell me the reasoning behind a score that is out of line with the table. My reasoning is that I need an answer when that team asks me "why?" once they've received their scoring packet. But, I also make sure they understand that I am not telling them how to judge and am not trying to tell them they are wrong in their scoring. If they felt that was the score they needed to give, so be it. But, be ready to explain.

Sorry for the long post.

Phillip

ZILLA
08-10-2010, 04:12 PM
Yes, as Sled suggested I'll move on.

Sledneck
08-10-2010, 04:12 PM
I could see only a handful of judges being lazy and scoring up. Reverse it, would a cook not try as hard on their entries if they though the judges were being lazy and giving higher scores? Doubt it. Everybody would still try their damn hardest to get that 9.

Sledneck
08-10-2010, 04:14 PM
Yes, as Sled suggested I'll move on.

Thank You , no matter what they say about you I think your a gentleman :thumb:

motoeric
08-10-2010, 04:18 PM
I agree wholeheartedly with the earlier post that a specific score mandating a comment card will engender a tendency to score above that number.

I think that the solution is to have more active encouragement from reps, table captains and organizers to give feedback and fill out comment cards.

I certainly point the finger at myself here, as I should be more proactive about this than I have been. I'll definitely try to encourage it more in events that I help to run in the future.

Eric

QueTPiesBBQ
08-10-2010, 04:19 PM
So the judge didn't like your sauce and told you why.....
what more do you want?
Ed


Mainly wondered why all the other judges at the table scored 7, 8, 9, 9, 8 and then the one scored a 5. Maybe just a bad judge, who knows. Comment didn't make much sense to me, of course at the time. After cooking all night and then to have someone put a comment like that on a card and a low score, makes you want to jump up and down.:boxing:

Sledneck
08-10-2010, 04:32 PM
I certainly point the finger at myself here, as I should be more proactive about this than I have been. I'll definitely try to encourage it more in events that I help to run in the future.

EricSo is it safe to say that we can blame you for the unpredictable and low scoring judges on lawnguyland?:wink:

Rookie'48
08-10-2010, 04:33 PM
Trust me on this one, folks - if I give you a 6 or lower then you'll more than likely get a comment card. We, as judges, are instructed that comment cards are strictly voluntary and that they should only be used for constructive criticism. The judges' CD and the little 'pep talk' that damn near all Reps give before we start judging are very plain on when a card "should" be filled out & what you should say. A comment like: "Tastes like crap" should earn that judge a trip to the wood shed. A card that says: "No flavor profile other than salt" might mean just the same thing, but at least you know why it tastes like crap.
I realize that one comment card at one contest isn't going to make you change your whole game plan - and it shouldn't! But if you get a couple of cards in two or three comps in a row & all of those cards say the same thing .....

motoeric
08-10-2010, 05:34 PM
If only an organizer up here would have the balls to sanction an FBA contest up here in the northeast, sigh...........


There's this guy named Steve from Long Island who is a big FBA fan. Maybe he'll step up. I doubt it, but maybe.

Eric

motoeric
08-10-2010, 05:35 PM
So is it safe to say that we can blame you for the unpredictable and low scoring judges on lawnguyland?:wink:

I'll take some responsibility for teams not getting comment cards more often, but low scores? Nah, not going to take responsibility for that.

Eric

Rub
08-10-2010, 06:58 PM
There's this guy named Steve from Long Island who is a big FBA fan. Maybe he'll step up. I doubt it, but maybe.

Eric
We're ready! :thumb:

Talking strictly FBA here, but I don't see the mandatory comment card rule we have for a 7 or lower making the judges avoid giving 7s and lower. I got 2 7s not long ago on the same entry, one said the smoke had a soured taste, the other said had a chemical taste. I thought about it, inspected my smoker, and realized it was in dire need of a deep cleaning. :doh: Our judges don't seem to be hesitant...

Sledneck
08-10-2010, 08:17 PM
....

Buster Dog BBQ
08-10-2010, 08:30 PM
We're ready! :thumb:

Talking strictly FBA here, but I don't see the mandatory comment card rule we have for a 7 or lower making the judges avoid giving 7s and lower. I got 2 7s not long ago on the same entry, one said the smoke had a soured taste, the other said had a chemical taste. I thought about it, inspected my smoker, and realized it was in dire need of a deep cleaning. :doh: Our judges don't seem to be hesitant...
Without those comments though you probably wouldn't have thought about it. That's good use of a comment card to let a cook know why a score was as it is. We got one that had a 5 for taste and they told us it was too salty. We had added extra rub that day so made sense. But most of the time it's a guessing game. Sigh.

deez20
08-10-2010, 09:23 PM
I wouldn't mind seeing a card from time to time for a low score. We took RGC at the battle of the bbq brethren over the weekend and looking at the scores we got a 4 from one judge in tenderness for pork with mostly 7,8 or 9 from the other judges.

deena
08-13-2010, 04:52 PM
In PNWBA we are rolling out comment cards and are trying to have them at all qualifier events. We ask that judges give comments for all scores. The comments are reviewed for inappropriate type comments (swear words and such) so we might weed out a couple. But judges and teams seem to like them. We are working on making sure that the comments are helpful and appropriate. The comments have actually helped us identify some judges that were judging incorrectly and some items so they have been very helpful!

Jeff S.
08-13-2010, 10:12 PM
I got comment cards from PNWBA in Seattle in the spring. Alot of them were very positive. 99 percent were all good stuff. Like to see this at all comps!

Keep it going PNWBA

Sawdustguy
08-14-2010, 06:06 PM
There's this guy named Steve from Long Island who is a big FBA fan. Maybe he'll step up. I doubt it, but maybe.

Eric

I would like to see a FBA contest here on LawnGuyLand also. I would also like to see a MBN contest here.

Sledneck
08-14-2010, 07:01 PM
I would like to see a FBA contest here on LawnGuyLand also. I would also like to see a MBN contest here.
Unfortunately we are the minority guy. I approached a few organizers who were planning new contests about sanctioning it FBA. The most common answer i received was they wouldn't take the chance of not sanctioning it KCBS because they think they wouldnt get the teams. I disagree, i think teams would be drawn to it because it was something different.

Sawdustguy
08-15-2010, 07:40 AM
I agree Steve! Something different is precisely why I think an FBA contest would attract teams. I can only speak for myself, but I would be first in line. I would also like to try a MBN contest. Michele tells me I am made of 99% Bullchit and 1% water.:thumb:

LindaM
08-15-2010, 09:56 PM
Absolutley untrue.... Who told you that this is the case or are you guessing that's what happens?
During judges instructions we are told that ALL cards get attached to your score sheets, period.

As far as mandatory comment cards at any given number, whatever one you choose (4,5,6) it puts a mental block that says if I give that number I have to fill out a card. Chances are you'll never see that number or lower again. Is that how you think judging should go?
Remember a 5 is just simply below average, not crap, not inedible, 5 is not that far out of whack. If your food is just slightly less than average your gonna get a 5. I have given out several 5's this year.
And as a rule I have filled out 4 comment cards also. They simply said what it was that made that sample a 5. Thats how the system works.
Tis true all judges don't always get it, but find anything in life that is just the way everbody wants it.
Ed

And first hand knowledge tells me that a certain BOD REP does not use comment cards at contests, "they can't be bothered with the extra work and the cooks don't want them anyway"

Alexa RnQ
08-15-2010, 11:41 PM
We cooked with three BOD reps this weekend and received two comment cards, one of which was quite detailed -- several sentences.

fesdwino
08-16-2010, 12:08 AM
As a newbie KCBS certified BBQ judge having just completed my first event, I agree with one comment that there should be more training on how to score BBQ. Knowing the rules is important, and I know judging BBQ, like wine, is somewhat subjective. It would be helpful to have a bit more guidance. Although I am a new judge and have much to learn, I am a student of BBQ and have cooked quality BBQ as well as taken a class from a winning BBQ competitor. I have a pretty good idea of what average to great BBQ is. That said, I may not judge the same as another judge.

I do believe we had explicit instruction on comment cards at both the judging class and the event. No need to comment on good scores as they speak for themselves. BBQ less than average deserves a constructive comment. The KCBS organizers at our event did a great job of keeping mixed levels of judging expertise (as measured by number of events judged) at each table. The Table Captains were diligent in their monitoring of the scores given, looking for unusual variances beween judges' scores and asking judges for comments when they saw a below average score. Every judge at our table had cooked on a competition team or had good BBQ experience and all took their job seriously.

Mandatory comment cards for scores below 6 would not be an issue for me and I don't believe most judges would object. I believe most judges are interested in being fair and objective.

Smokenstein & monster crew
08-16-2010, 04:38 PM
theres always that one score/judge that you just have to say WTF? compared to the others, thinking something must have been amiss & a comment card would greatly relieve the questions running circles in my head.....

Slamdunkpro
08-16-2010, 04:54 PM
Here's an idea instead of tying the comment card to a specific score, have the table pull the low score of the six regardless of the numbers and help the judge write a comment card for that score. That way every team gets 4 comment cards describing the reasoning behind the lowest score at the table.

ModelMaker
08-16-2010, 05:08 PM
As a newbie KCBS certified BBQ judge having just completed my first event, I agree with one comment that there should be more training on how to score BBQ. Knowing the rules is important, and I know judging BBQ, like wine, is somewhat subjective. It would be helpful to have a bit more guidance. Although I am a new judge and have much to learn, I am a student of BBQ and have cooked quality BBQ as well as taken a class from a winning BBQ competitor. I have a pretty good idea of what average to great BBQ is. That said, I may not judge the same as another judge.

I do believe we had explicit instruction on comment cards at both the judging class and the event. No need to comment on good scores as they speak for themselves. BBQ less than average deserves a constructive comment. The KCBS organizers at our event did a great job of keeping mixed levels of judging expertise (as measured by number of events judged) at each table. The Table Captains were diligent in their monitoring of the scores given, looking for unusual variances beween judges' scores and asking judges for comments when they saw a below average score. Every judge at our table had cooked on a competition team or had good BBQ experience and all took their job seriously.

Mandatory comment cards for scores below 6 would not be an issue for me and I don't believe most judges would object. I believe most judges are interested in being fair and objective.

I think you have a pretty good handle on whats going on. The first year you are indeed just a rookie. Pay attention, don't pretend you know it all. Give judging your full attention and walk before you run.
Talk among your table after scores are in and ask questions, compare scores. Here is your best area to increase knowledge. After a while you'll know which judges you can learn from vs the blowhards. But by all means cultivate your own style and practices.
I've been at this for 5 years now and am just starting to feel comfortable with the whole process. I'm confident in my decisions but do not have iron clad opinons, you must change thought process as you grow your ideals and experiance.
As soon as you know everything and decide your the best judge.. quit, because your not growing and learning and that's a neccesity in this endeavor.
Ed

Lake Dogs
08-16-2010, 05:29 PM
I think you have a pretty good handle on whats going on. The first year you are indeed just a rookie. Pay attention, don't pretend you know it all. Give judging your full attention and walk before you run.
Talk among your table after scores are in and ask questions, compare scores. Here is your best area to increase knowledge. After a while you'll know which judges you can learn from vs the blowhards. But by all means cultivate your own style and practices.
I've been at this for 5 years now and am just starting to feel comfortable with the whole process. I'm confident in my decisions but do not have iron clad opinons, you must change thought process as you grow your ideals and experiance.
As soon as you know everything and decide your the best judge.. quit, because your not growing and learning and that's a neccesity in this endeavor.
Ed

feswino, I think most competitors would LOVE to have someone like you
judge their entries.

Ed, I agree with you 100%. However, the only judges that I've ever
seen that know it all seem to know it all on that 3rd or 4th event. Soon
they'll see something new or different that rocks their world... From what
I've seen anyway. Same 5+ years, and I'm still seeing different things
and learning.

In KCBS scoring, I, personally, would like to see constructive criticism for
anything 7 or below, and I think one could easily say anything not a 9
deserves constructive criticism. Others have construed this to mean
folks should just get 9's and go on. I didnt mean that at all. However, I
think as much feedback as possible is due, even if the cook and the judge
disagree. At least the cook understands what he/she is dealing with.

JMHO.

route66
08-21-2010, 01:49 AM
While I am new at this competition stuff having only a couple competitions experience, How does one figure out judging? I am hopeful that others have received as I, 9-9-6 or 9-8-6 without any comment card. Hey, if it sucks and deserves a 6 tell me as I and a few other judges missed it. I enjoy constructive criticism as it will make me become better. All 6's should have a comment in my opinion.

ModelMaker
08-21-2010, 04:17 PM
While I am new at this competition stuff having only a couple competitions experience, How does one figure out judging? I am hopeful that others have received as I, 9-9-6 or 9-8-6 without any comment card. Hey, if it sucks and deserves a 6 tell me as I and a few other judges missed it. I enjoy constructive criticism as it will make me become better. All 6's should have a comment in my opinion.


See now here is part of the problem
" Hey, if it sucks and deserves a 6 tell me "
6 doesn't mean SUCKS, 6 means it's average!!!
7 means the first number above average
5 is the first number below average

In my opinion if your entry is just a pinch below what I consider average you may indeed get a 5.
Remember 5 is only the first step below average.
I am getting the impression from many of you cooks that a 6 means (to you) I thought your food sucks and I don't think that is the case.
Apparantly you think anything short of a 8 or 9 is a slap in the face.
Not true, 8 is very good, 9 is reserved for excellence.
If anything I think scores (myself included) have been running a little high lately. During table talk I've had a judge say sample 4 was fantastic I gave it a 9 and looking back I never would have considered it a 9.
This comment card system is still in it's infancy, make your thoughts known to the board members and someday maybe it will be what both sides want.
Ed

BrisketBelly
08-21-2010, 05:20 PM
I judged a comp today and we received 5 entries of ribs at our table. After we were done with the ribs, we had a discussion about it and all the judges were in line with one another that every entry was merely average/above average. I personally felt one rib was cooked perfectly and deserved a 9 for tenderness. A short time later, the contest rep came to our table to discuss how everyone had scored 7s for nearly all the entries and how one of the judges did not score a single 8 or 9 on their card. We were all thrown by him questioning our scoring, especially since they apparently were pretty darn close to one another. He said something about how we should want to matter in scoring since the lowest score is thrown out and if we all put 7s, then our scores wont mean anything. I gotta tell ya, I don't understand his intentions and wonder why he would even bring that up to us.

On a side note - I left a comment card today for something as I felt it would that help team. If you are doing it, I hope you consider it as well. A chicken entry used shredded lettuce as the base of their garnish. With the juices and sauce on the chicken, you can imagine what the bottom of the chicken was like. It certainly left a lettuce taste on the chicken as well.

Slamdunkpro
08-21-2010, 06:33 PM
.... A short time later, the contest rep came to our table to discuss how everyone had scored 7s for nearly all the entries and how one of the judges did not score a single 8 or 9 on their card. We were all thrown by him questioning our scoring, especially since they apparently were pretty darn close to one another. He said something about how we should want to matter in scoring since the lowest score is thrown out and if we all put 7s, then our scores wont mean anything. I gotta tell ya, I don't understand his intentions and wonder why he would even bring that up to us.

Did the rep come over and say this before the pork or brisket turn-in or after all judging was over????? If they did it before judging was over that smacks of tampering.

carlharper
08-21-2010, 06:59 PM
BrisketBelly

As you described the circumstances, it seems that the reps comments were way out of line. Did any of you challenge his criticism or were you too surprised at the comments to mount a defense? It sounds like your scores were well within acceptable variances, assuming that there wasn't one judge consistently scoring lower than all the others. Very interesting.

BrisketBelly
08-21-2010, 07:00 PM
Did the rep come over and say this before the pork or brisket turn-in or after all judging was over????? If they did it before judging was over that smacks of tampering.

It was between. None of us changed scores nor did he ask us to.

Slamdunkpro
08-21-2010, 07:23 PM
It was between. None of us changed scores nor did he ask us to.
Not on the ribs, but the rep's (and it was the rep, not the table captain?) comments had to be in your mind when your table judged pork and brisket. Those comments might have been enough to turn some judge's pork & brisket 7's into 8's or 9's. Those comments could easily change your table from "tough but fair" to "generous" boosting teams that got you for pork and brisket.

ModelMaker
08-22-2010, 12:31 PM
I judged a comp today and we received 5 entries of ribs at our table. After we were done with the ribs, we had a discussion about it and all the judges were in line with one another that every entry was merely average/above average. I personally felt one rib was cooked perfectly and deserved a 9 for tenderness. A short time later, the contest rep came to our table to discuss how everyone had scored 7s for nearly all the entries and how one of the judges did not score a single 8 or 9 on their card. We were all thrown by him questioning our scoring, especially since they apparently were pretty darn close to one another. He said something about how we should want to matter in scoring since the lowest score is thrown out and if we all put 7s, then our scores wont mean anything. I gotta tell ya, I don't understand his intentions and wonder why he would even bring that up to us.

On a side note - I left a comment card today for something as I felt it would that help team. If you are doing it, I hope you consider it as well. A chicken entry used shredded lettuce as the base of their garnish. With the juices and sauce on the chicken, you can imagine what the bottom of the chicken was like. It certainly left a lettuce taste on the chicken as well.


So this impartial KCBS rep admonished your table for scoring all the rib entries as "above average"?
That is improper on several levels.
If it were me I would drop an e-mail to the appropriate board member.
Ed

Slamdunkpro
08-22-2010, 01:25 PM
Should this discussion be split into it's own thread?

pop's smokin hot que
08-22-2010, 08:37 PM
Wow what a subject! After reading the discussion, here is my take. With 6 being average, not every entry is going to be 999. I have judged 10 events so far this year. I always ask the table captain if the table is pretty close on scoring. We are usually within 2 points for the 6 judges. Yesterday I gave a pork box a 9 for apperance. The next two scores were 7, 6. It sure looked good, all the taste was in the sauce not the meat. The money meat was so tough you could not tear a bite out thus the 6. As a past competitor I know the cook knew it was tough as shoe leaather when he turned it in. You know if it's good or below average when you put it in the box. I didn't drive 200 miles at my expense just to screw a cook. My advise is get over it or get better.

Black Cat BBQ
08-23-2010, 06:30 AM
Score sampling from Dover:

Ribs Taste: 995788
Pork Tenderness: 794787

Being a 2nd year team we know we have a long way to go so we are fine with 6's, 7's and 8's. What I am not fine with is the 4 and 5 and no comment card. WTH? I make sure only the best pieces go in my boxes (like everyone else) so short of a rock getting in there I have no clue how judge 3 thought to give us a 4. A comment card would go a long way on that one.

Ribs maybe because we only put 6 in, maybe one did taste off? Dont know and they are always from the same rack.

Congrats to Tuffy and all who heard their names called!! :clap2: We love this comp and will be back next year. Hopefully sans 4's and 5's. :-P

Muzzlebrake
08-23-2010, 10:47 AM
what would you guys do if you got a 2 on your scorecard?

HawgHeaven
08-23-2010, 12:02 PM
Score sampling from Dover:

Ribs Taste: 995788
Pork Tenderness: 794787

Being a 2nd year team we know we have a long way to go so we are fine with 6's, 7's and 8's. What I am not fine with is the 4 and 5 and no comment card. WTH? I make sure only the best pieces go in my boxes (like everyone else) so short of a rock getting in there I have no clue how judge 3 thought to give us a 4. A comment card would go a long way on that one.

Ribs maybe because we only put 6 in, maybe one did taste off? Dont know and they are always from the same rack.

Congrats to Tuffy and all who heard their names called!! :clap2: We love this comp and will be back next year. Hopefully sans 4's and 5's. :-P
Comment cards are always welcome in my camp... I need to know what the judges think of my food. Should they be manditory? No, for many reasons already mentioned.

Jen, thanks a million for letting my wife and I hang out with you folks at Dover! You guys ROCK!! :thumb:

PimpSmoke
08-23-2010, 12:52 PM
what would you guys do if you got a 2 on your scorecard?

Take my ball and go home.

Sledneck
08-23-2010, 12:52 PM
what would you guys do if you got a 2 on your scorecard?

I would quit:becky:

Smokedelic
08-23-2010, 03:03 PM
what would you guys do if you got a 2 on your scorecard?
Cook again the next weekend and not change a thing. Probably would take that contest off my schedule for next year though.

I got a 2, in appearance of all things, from a judge in 2007 in Great Bend, KS. Asked the Rep to take a look and see if it was an entry error, which they refused.

Haven't been back to Great Bend since. I've got nothing against the contest, the organizer, or the Rep. I have other places I can go cook, and they have other teams to take my place. It works well for everyone:thumb:.

Muzzlebrake
08-23-2010, 08:10 PM
I was just shocked (still am really) that there was no explanation at all. It was in the shrimp category during the grilling contest......I think it gets me off the hook for future pot lucks....:thumb:

BigBarry
08-23-2010, 09:23 PM
As a former computer guy, everything is data and the data should be collected after each contest and sent to KCBS. Each judge should be analyzed (by a formula) to see if they are consistently low or high on each entry they judge (by a certain factor). If they are giving low (or even high) scores across each category, the KCBS should contact them and see why they are so off from the rest of the judges.

If the judge just tanked just one category vs. the rest of the table, maybe give them a pass.

This way you can weed out/adjust judges (if this is a systemic problem.)

Judging is subjective and YES - they drop the lowest score but if that lowest score was higher, then maybe the overall would be higher.

I am throwing the ball into the KCBS yard...

Just my 2c.

Sledneck
08-23-2010, 10:09 PM
As a former computer guy, everything is data and the data should be collected after each contest and sent to KCBS. Each judge should be analyzed (by a formula) to see if they are consistently low or high on each entry they judge (by a certain factor). If they are giving low (or even high) scores across each category, the KCBS should contact them and see why they are so off from the rest of the judges.

If the judge just tanked just one category vs. the rest of the table, maybe give them a pass.

This way you can weed out/adjust judges (if this is a systemic problem.)

Judging is subjective and YES - they drop the lowest score but if that lowest score was higher, then maybe the overall would be higher.

I am throwing the ball into the KCBS yard...

Just my 2c.
such a thing exists its called FBA

ALX
08-23-2010, 10:52 PM
such a thing exists its called FBA



I like the idea of some sorta judge policing etc...

BigBarry
08-24-2010, 09:55 AM
such a thing exists its called FBA


I guess the FBA is more concerned about judging quality if they proactively do something like this.

It's sad that the largest BBQ sanctioning organization (KCBS) doesn't see the benefit of monitoring judging quality/understanding.

Muzzlebrake
08-24-2010, 10:22 AM
happened in the NEBS contest not KCBS. NEBS doesn't have a judging certification that I know of.

mfreeman73
08-24-2010, 11:11 AM
As a former computer guy, everything is data and the data should be collected after each contest and sent to KCBS. Each judge should be analyzed (by a formula) to see if they are consistently low or high on each entry they judge (by a certain factor). If they are giving low (or even high) scores across each category, the KCBS should contact them and see why they are so off from the rest of the judges.

If the judge just tanked just one category vs. the rest of the table, maybe give them a pass.

This way you can weed out/adjust judges (if this is a systemic problem.)


I had been thinking about something like that and would love to see it implemented. If a judge consistently scores much lower than his fellow judges at his table, then after a while KCBS could send him a letter saying, "You know, you really suck as a judge. We're going to require you to take the CBJ class again before you torture any more cooks."

And I'm talking about someone who does it consistently, not just the rogue here and there.

And I agree with whoever said there needs to be spent more time in the CBJ class going over what is average, what is good, and what isn't good. I remember when I took the class, after we tasted and scored some meats, the instructor would ask what everyone's scores were. You'd have some that scored it 3's and 4's (and the meat was clearly not that low). These are the judges that are probably going to be a problem.

And heck, I'm not against making CBJ's cook with a team before judging. I know that's harsh, but it would probably weed out some of the bad judges and also help a lot of them understand what goes into cooking this food.

Jorge
08-24-2010, 11:35 AM
I judged a comp today and we received 5 entries of ribs at our table. After we were done with the ribs, we had a discussion about it and all the judges were in line with one another that every entry was merely average/above average. I personally felt one rib was cooked perfectly and deserved a 9 for tenderness. A short time later, the contest rep came to our table to discuss how everyone had scored 7s for nearly all the entries and how one of the judges did not score a single 8 or 9 on their card. We were all thrown by him questioning our scoring, especially since they apparently were pretty darn close to one another. He said something about how we should want to matter in scoring since the lowest score is thrown out and if we all put 7s, then our scores wont mean anything. I gotta tell ya, I don't understand his intentions and wonder why he would even bring that up to us.

On a side note - I left a comment card today for something as I felt it would that help team. If you are doing it, I hope you consider it as well. A chicken entry used shredded lettuce as the base of their garnish. With the juices and sauce on the chicken, you can imagine what the bottom of the chicken was like. It certainly left a lettuce taste on the chicken as well.

Were the scores for the table similar for all of the other categories?

If the other tables were scoring higher, I think the Rep had cause to come talk to the table. If it happened consistently over multiple categories then I REALLY think the Rep had cause to have a word with the table.

For a team that is contending, 7's can be a killer. If the product deserved a 7 then so be it.

ModelMaker
08-24-2010, 11:59 AM
"For a team that is contending, 7's can be a killer. If the product deserved a 7 then so be it."

I think this mentality is the biggest scoring problem.
"If the product deserves a 7 (above average) then so be it"...

7 is NOT a bad score, maybe there are too many 9's being handed out that should be reserved for "excellent".
I'm all for further standardized judges training ( free or very reasonable rates) and work on what seperates above average from excellent. It still must be all about each judges personal decision but within some sort of guidlines.
It all depends on wether KCBS wants to be the premire BBQ promoter in the land, or just muddle through with what they have.
Ed

Smokedelic
08-24-2010, 12:53 PM
If a judge consistently scores much lower than his fellow judges at his table, then after a while KCBS could send him a letter saying, "You know, you really suck as a judge. We're going to require you to take the CBJ class again before you torture any more cooks."

So you would also agree that KCBS should track cooking teams scores, and for those that score consistently low, KCBS should send them a letter saying "You know, you really suck as a cook. We're going to require you to take a cooking class before you torture any more judges"?

billm
08-24-2010, 12:54 PM
I welcomed comment cards when they first started them then found out they were just as frustrating as the score themselves and didnt learn anything from them
case in point at one contest
Card 1: brisket nice and tender
Card 2: Brisket very tough
Both cards from same table same brisket
so not sure what making them mandatory for low scores would do

mfreeman73
08-24-2010, 01:58 PM
So you would also agree that KCBS should track cooking teams scores, and for those that score consistently low, KCBS should send them a letter saying "You know, you really suck as a cook. We're going to require you to take a cooking class before you torture any more judges"?

I actually got a letter from KCBS saying pretty much that. They banned me from any further comps until I cook a brisket correctly. :-P

Actually, it's apples and oranges. The teams spend lots of money to compete and all they want is to be judged fairly. And if I cook bad food, it's only hurting myself. It's not like I just wasted the judge's money.

But, if a judge gives low scores to everything, or lower than they reasonably should be, then they are wasting the time and money of the competitors. And I'm not talking about someone who scores something a 6 when others would usually give it a 7 or 8. I'm talking about someone who would score something a 4 when others are giving it a 7 or 8.

That would actually be fine if that judge was judging every competitors food, but that doesn't happen. If you had a decent turn-in and you got a table with a bad judge, that judge could really wreck your scores.

Yeah, I know that's just part of the game, but I think what we want to do is minimize that as much as possible. I don't think you could get rid of it altogether, but maybe there are some steps to help educate judges a little more so that if you turn in something that really should be scored an 8, then the six scores should average somewhere around there.

Sawdustguy
08-24-2010, 02:34 PM
I don't think comment cards are the answer either. There are those who judge and just don't give a chit. They could care less. To them it's just a number on a card. They may as well go eeny-meany-miney-mo. Thank goodness they are in the vast minority but they seem to show up at every contest. Do you think that the judges that are not willing to give you are fair shake when tasting your turn-in are going to actually write anything meaningful on a comment card?

fesdwino
08-24-2010, 02:56 PM
Thanks for the kind words. If I was competing I'd like to get as much feedback as I could.

KC_Bobby
08-24-2010, 04:25 PM
"For a team that is contending, 7's can be a killer. If the product deserved a 7 then so be it."

I think this mentality is the biggest scoring problem.
"If the product deserves a 7 (above average) then so be it"...

7 is NOT a bad score, maybe there are too many 9's being handed out that should be reserved for "excellent".
Ed

Ed, I'm not agreeing nor disagreeing with your comment but I think for the cooks that go out there regularly a 7 is not a score we like to see. As Jorge said, we can't afford them - especially in IA where scores have been off the charts this year. (In Marshalltown a 170 rib wasn't even top 10).

Wins in Iowa are taking an average of 674+ points - that's about an average per score score of 33.7 per judge per turn in (997, 798, 599 or 989).

Again, not saying a 7 isn't a 7 - but with the vast range of scores from judges it really does help to land on the right table. It can help as much as it can hurt to land on a different one.

DawgPhan
08-24-2010, 05:32 PM
So you would also agree that KCBS should track cooking teams scores, and for those that score consistently low, KCBS should send them a letter saying "You know, you really suck as a cook. We're going to require you to take a cooking class before you torture any more judges"?


I would 100% be for having to earn your "pro card" like in golf, or qualify for events. whatever system they came up with. Enough with the free for all, time to tighten up everything from the cooks to the contests.

PimpSmoke
08-24-2010, 06:10 PM
I don't know when I saw one judge 999'd on my Brisket score last weekend I danced a farkin' jig.

Of course 5's chap my ass, but I didn't get any of those last week.

Jorge
08-25-2010, 07:59 AM
Mod Note: There are several different threads currently running within this one, that are all worthy of discussion. Any objection if I split some of those off later today?

Jorge
08-25-2010, 08:17 AM
"For a team that is contending, 7's can be a killer. If the product deserved a 7 then so be it."

I think this mentality is the biggest scoring problem.
"If the product deserves a 7 (above average) then so be it"...

7 is NOT a bad score, maybe there are too many 9's being handed out that should be reserved for "excellent".
I'm all for further standardized judges training ( free or very reasonable rates) and work on what seperates above average from excellent. It still must be all about each judges personal decision but within some sort of guidlines.
It all depends on wether KCBS wants to be the premire BBQ promoter in the land, or just muddle through with what they have.
Ed

Ed, I understand your point. It seems to me that scores are up this year, but I haven't had the time to sit down and crunch the numbers to confirm that. Maybe a lot more teams are cooking that much better, I don't know.

If it's a 7, then it should be scored a 7. I just hope that every judge that scores an entry a 7 fully understands the impact that can have on a team. While you may view a 7 as a good score, it can kill the chances of a team in contention for a GC and that's a fact. Bob pointed out the impact it has on the score. At the top end of the field, this year, the contests are that close. I don't think anybody is asking the judges to start at 8 or 9, but I don't think the bar should be set so high by a judge as to make getting that 8 or 9 virtually unobtainable.

Slamdunkpro
08-25-2010, 08:39 AM
If it's a 7, then it should be scored a 7. I just hope that every judge that scores an entry a 7 fully understands the impact that can have on a team. While you may view a 7 as a good score, it can kill the chances of a team in contention for a GC and that's a fact.
It's not in my job description as a judge to consider this.

My job as a judge is to evaluate the sample placed in front of me as it is presented. Nothing more. That the cooks might have lavishly basted the flown in fresh from over-there-a-stan meat using ostrich feathers every 15 minutes all night long after driving 36 hours and spending $$$$$ to attend this competition should not, no, must not influence my evaluation, just as the possibility that my score might not "matter" or might harm the competitor's chances at getting an award.
It's all about the end product, as it's presented to me, judged under the KCBS guidelines - period.

KC_Bobby
08-25-2010, 09:11 AM
This is a tricky subject, one which currently doesn't have an answer. I don't fault judges for sticking by their scoring system and it's very possible that 8s and 9s are handed out too often. Which makes the 7 hurt even more. But at the same time, if a judge is aware that he/she may have a strict/lenient scoring system compared to other judges and has an understanding of how that could impact the entries on which they judge - wouldn't it be something for that judge to evaluate within themselves away from the judging table?

The difference in scores using the following for average scores:
An average 9 is a 36 per judge, x 5 judges = 180 x 4 entries = 760 (perfect comp score)
An average 8 is a 32 per judge, x 5 judges = 160 x 4 entries = 640 (not a bad day, but would likely finish no higher than 5th in the Midwest - possibly not in the overall Top 10)
An average 7 is a 28 per judge, x 5 judges = 140 x 4 entries = 560 (sever disappointment)
An average 6 is a 24 per judge, x 5 judges = 120 x 4 entries = 480 (likely DAL)

I'd really like to see KCBS track judging scores because scores are subjective. What one person sees as a 7 another may see as a 9. Neither individual is necessarily wrong. But having a system that puts an level judging score history per table would balance differences.

PimpSmoke
08-25-2010, 09:18 AM
So what about this one? When I write an employee evaluation I have to justify bad and good.

If the judges just throwing around nines would have to JUSTIFY that nine on a card with a comment, less judges would just throw out that number.

Want to give it an excellent score? Fine, tell the rep and the team why they deserve that score.

Same for low scores.

camano
08-25-2010, 09:19 AM
It's not in my job description as a judge to consider this.

My job as a judge is to evaluate the sample placed in front of me as it is presented. Nothing more. That the cooks might have lavishly basted the flown in fresh from over-there-a-stan meat using ostrich feathers every 15 minutes all night long after driving 36 hours and spending $$$$$ to attend this competition should not, no, must not influence my evaluation, just as the possibility that my score might not "matter" or might harm the competitor's chances at getting an award.
It's all about the end product, as it's presented to me, judged under the KCBS guidelines - period.

I agree with your comment. Judging each entry on its own merits and being consistent. If any judge gives a score that is more than a couple of points lower or higher than the table average the table captain should find out why and ensure the judging guidelines are being followed.

PimpSmoke
08-25-2010, 09:24 AM
I'd really like to see KCBS track judging scores because scores are subjective. What one person sees as a 7 another may see as a 9. Neither individual is necessarily wrong. But having a system that puts an level judging score history per table would balance differences.

Not only that, but it squashes any possible collusion. If judges start "putting their mark" on their scores I think there would be HUGE changes in numbers.

Slamdunkpro
08-25-2010, 09:42 AM
Not only that, but it squashes any possible collusion. If judges start "putting their mark" on their scores I think there would be HUGE changes in numbers.
Judges are already supposed to sign their scorecards

Slamdunkpro
08-25-2010, 09:47 AM
I'd really like to see KCBS track judging scores because scores are subjective. What one person sees as a 7 another may see as a 9. Neither individual is necessarily wrong. But having a system that puts an level judging score history per table would balance differences.
How can you do this? Not every sample at every event is a 7,8,or 9. If you hit 3 or 4 tables over 3 or 4 events as a judge where the samples you get aren't good and you score accordingly then your score average as a judge goes down.

Jorge
08-25-2010, 09:53 AM
How can you do this? Not every sample at every event is a 7,8,or 9. If you hit 3 or 4 tables over 3 or 4 events as a judge where the samples you get aren't good and you score accordingly then your score average as a judge goes down.

Easily, if the judges #(if a CBJ) was entered during data entry.

Edit: The analysis of the data after just a few contests would be pretty revealing, and over time would paint a pretty clear picture.

You just really messed up my day, since I write code for a living!

PimpSmoke
08-25-2010, 10:03 AM
Judges are already supposed to sign their scorecards

Understood, but not my point. Does someone actively track this?

Does anyone look to see if Judge69 somehow had Team69 land on his/her table more than a few times over the years and compare the scores?

I think ithe results of that would be very interesting.

KC_Bobby
08-25-2010, 10:08 AM
How can you do this? Not every sample at every event is a 7,8,or 9. If you hit 3 or 4 tables over 3 or 4 events as a judge where the samples you get aren't good and you score accordingly then your score average as a judge goes down.

Good point and I sure won't claim that I have the answers (not meaning that was implied) but maybe some thoughts will brainstorm into better ideas.

Maybe the scoring system per judge is based on a +/- score based on the table average score. Complex yes, but smart people can write that data entry code fairly easily.

Slamdunkpro
08-25-2010, 10:13 AM
Easily, if the judges #(if a CBJ) was entered during data entry.

Edit: The analysis of the data after just a few contests would be pretty revealing, and over time would paint a pretty clear picture.

You just really messed up my day, since I write code for a living!
Your assuming that the entries are a constant - they aren't.

Smokedelic
08-25-2010, 10:25 AM
If it's a 7, then it should be scored a 7. I just hope that every judge that scores an entry a 7 fully understands the impact that can have on a team. While you may view a 7 as a good score, it can kill the chances of a team in contention for a GC and that's a fact. Bob pointed out the impact it has on the score. At the top end of the field, this year, the contests are that close. I don't think anybody is asking the judges to start at 8 or 9, but I don't think the bar should be set so high by a judge as to make getting that 8 or 9 virtually unobtainable.

If a team turns in something that could be scored a 7, then they aren't in contention for a GC anyway. 8s and 9s aren't unattainable, because the best cooks are getting them every week.

IMO, if a cook wants 8s and 9s, then they need to learn how to cook and present everything PERFECTLY....every week, because that's what the best cooks do week in and week out. The best cooks take "table luck" out of the equation by consistently cooking and presenting everything better than the rest of us.

So the way I see it, you can either complain about judges, and KCBS not tracking judges scores, etc. or you can focus on perfecting your craft...because if you do that, the rest of it won't matter.

JMO...

mfreeman73
08-25-2010, 10:44 AM
So the way I see it, you can either complain about judges, and KCBS not tracking judges scores, etc. or you can focus on perfecting your craft...because if you do that, the rest of it won't matter.

I think that's what it ultimately comes down to. If a team is consistently at the top, is it because they get lucky judge's tables all the time? No, it's more that they just cook some good food.

The system isn't perfect, but it isn't bad either. I think they try to help out with the "bad judge" situation by dropping the lowest score. That way if five judges at your table scores you 8's and 9's and one gives you a 5, well their score won't count anyway (unless there's a tie).

I really only see two options to help with this:
1. More education for judges
2. Keeping track of judge's scores and addressing those that are way out of line with the other judges at their table.

Both would take more time and money, so they're not easy solutions. I think as judges do more competitions, they get better about deciding what is good and what is not so good. I know I judge a bit better now than when I did my first comp. That first one I had no clue. And I bet a lot of judges are like that.

Maybe if someone doesn't do but maybe one contest a year or something like that they have to retake the CBJ class every so often. Kind of like saying that the way to not have to keep retaking the CBJ class is to do more judging. So, judges would either be getting more education by retaking the CBJ class or judging more competitions. Anyway, just throwing ideas out there.

KC_Bobby
08-25-2010, 10:55 AM
So the way I see it, you can either complain about judges, and KCBS not tracking judges scores, etc. or you can focus on perfecting your craft...because if you do that, the rest of it won't matter.


Perfect your craft and a team won't find a table that gives out less than 8s?

Here's an example from a score sheet I have from this season:

GC Team
C - 173.1428
R - 172.5714
P - 173.1430
B - 168.0000
O - 686.8572

RGC Team
C - 162.8570
R - 169.7142
P - 153.1428
B - 174.2856
O - 659.9996

The big difference their was the pork scores, lets look at them closer:

GC team - 35.4286 30.2858 27.4286 35.4286 36.0000 36.0000
RGC team- 26.2856 32.0000 33.7142 32.5714 28.5716 24.5716

GC's team had fairly consistent pork scores other than judge 3
RGC's pork scores were all over the board - 3 decent scores, an 877 and 2 hideous scores

Let's hypothetically assume that the two pork entries were close to equal in quality then trade the tables the two entries hit. (as I've heard from both cooks that two recipes are somewhat similar)

All of a sudden the RGC's overall score becomes 679.8570 and the GCs score becomes 666.8570

While I will not mention who the two teams are, both historically do well in pork.

Alexa RnQ
08-25-2010, 11:00 AM
If a team turns in something that could be scored a 7, then they aren't in contention for a GC anyway.
We received a 6 on our first-place brisket this past weekend, where we GC'd.

True, we're still perfecting our craft, but that does not mean the judging system is without flaws.

DawgPhan
08-25-2010, 11:04 AM
We received a 6 on our first-place brisket this past weekend, where we GC'd.

True, we're still perfecting our craft, but that does not mean the judging system is without flaws.


and they still didnt spell your team name right on the website. Congrats on the win.

Alexa RnQ
08-25-2010, 11:06 AM
Thanks! Hell, we've barely got our usual reps trained on the spelling. :becky:
< /threadjack >

Muzzlebrake
08-25-2010, 11:08 AM
I think that's what it ultimately comes down to. If a team is consistently at the top, is it because they get lucky judge's tables all the time? No, it's more that they just cook some good food.

The system isn't perfect, but it isn't bad either. I think they try to help out with the "bad judge" situation by dropping the lowest score. That way if five judges at your table scores you 8's and 9's and one gives you a 5, well their score won't count anyway (unless there's a tie).

I really only see two options to help with this:
1. More education for judges
2. Keeping track of judge's scores and addressing those that are way out of line with the other judges at their table.

Both would take more time and money, so they're not easy solutions. I think as judges do more competitions, they get better about deciding what is good and what is not so good. I know I judge a bit better now than when I did my first comp. That first one I had no clue. And I bet a lot of judges are like that.

Maybe if someone doesn't do but maybe one contest a year or something like that they have to retake the CBJ class every so often. Kind of like saying that the way to not have to keep retaking the CBJ class is to do more judging. So, judges would either be getting more education by retaking the CBJ class or judging more competitions. Anyway, just throwing ideas out there.

I agree with just everything you said, and I think at least in the Northeast, teams are cooking better to make for higher scores and KCBS scores seem much more consistent, which I attribute to more experienced judges.

One thing I am curious about is, as cooks we have our score to use as a benchmark or standard for bettering our performance. What benchmark/threshold/standard can judges use? How does a judge, judge their performance?

Alexa RnQ
08-25-2010, 11:10 AM
I think they try to help out with the "bad judge" situation by dropping the lowest score.
...
I really only see two options to help with this:
1. More education for judges
2. Keeping track of judge's scores and addressing those that are way out of line with the other judges at their table.
I heartily agree continuing education at periodic intervals for judges, and would even go so far as to wish for better quantification at training of what exactly constitutes a particular numerical score for each aspect of the four meats.

I would add one more factor in the table composition, though: the number of new/inexperienced judges. What we often see out here is organizers who rely on a Thursday-night class to fill their judging tables come Saturday morning. After a low score is thrown out, you've still got wildcards sitting on that table. I understand that's likely less of a factor in areas that are better supplied with experienced judges.

KC_Bobby
08-25-2010, 11:17 AM
After a low score is thrown out, you've still got wildcards sitting on that table. I understand that's likely less of a factor in areas that are better supplied with experienced judges.

I think this is exactly what happened in my example above as the CBJ percentage was low for the area.

Muzzlebrake
08-25-2010, 11:17 AM
I heartily agree continuing education at periodic intervals for judges, and would even go so far as to wish for better quantification at training of what exactly constitutes a particular numerical score for each aspect of the four meats.

I would add one more factor in the table composition, though: the number of new/inexperienced judges. What we often see out here is organizers who rely on a Thursday-night class to fill their judging tables come Saturday morning. After a low score is thrown out, you've still got wildcards sitting on that table. I understand that's likely less of a factor in areas that are better supplied with experienced judges.

In talking with organizers this year it seems that we may be past that point in our neck of the woods. This past weekend, we had over 100 experienced CBJ's at a 70 team contest. Contests seem to now be filling up the judges seats long in advance of the contest.

Jorge
08-25-2010, 11:40 AM
Your assuming that the entries are a constant - they aren't.

No, I'm not. I'm assuming that the judges are judging the entries as presented and doing the comparison against other judges over time. A bad entry or two will skew results, but in time they will even out. The consistent judges become a benchmark.

Smokedelic
08-25-2010, 11:43 AM
We received a 6 on our first-place brisket this past weekend, where we GC'd.

True, we're still perfecting our craft, but that does not mean the judging system is without flaws.
Congrats on your win last weekend, that was huge! I'm guessing your 6 was a score that was dropped, but maybe not. When you beat a field like that by 6 or 7 points, I'd say your craft was pretty close to perfect that day.:thumb:

FWIW, I never said the judging system is flawless, but I can't control that. I can only control how I cook, and I choose to focus on that.

Bentley
08-25-2010, 05:52 PM
Judges are already supposed to sign their scorecards

I have never been told to sign a KCBS score card in 8 years. I am told to put my name on it, maybe that is what you mean.

BigBarry
08-25-2010, 07:54 PM
How do KCBS members put in a motion for something like judge tracking and feedback?

Jorge
08-26-2010, 08:29 AM
How do KCBS members put in a motion for something like judge tracking and feedback?

I'd send the proposal to the BoD.

Warthog
08-26-2010, 08:47 AM
Judges are already supposed to sign their scorecards
No they just print their name on it.

Slamdunkpro
08-26-2010, 09:12 AM
No they just print their name on it.
Sign/print whatever - point was they are identifiable.

mobow
08-26-2010, 12:46 PM
I wish they would give us judges a print out of our table scores so we could see how we scored compared to the other judges at my table. The table captain will tell us if we were close to one another but that is about all the feed back we get. We don't get to see each others score card. As soon as it is turned into the table captain they are removed from the table. I think this would be a great learning tool for us judges.

goodsmokebbq
08-26-2010, 12:51 PM
I wish they would give us judges a print out of our table scores so we could see how we scored compared to the other judges at my table. The table captain will tell us if we were close to one another but that is about all the feed back we get. We don't get to see each others score card. As soon as it is turned into the table captain they are removed from the table. I think this would be a great learning tool for us judges.


Not a bad idea...

mfreeman73
08-26-2010, 01:50 PM
I wish they would give us judges a print out of our table scores so we could see how we scored compared to the other judges at my table. The table captain will tell us if we were close to one another but that is about all the feed back we get. We don't get to see each others score card. As soon as it is turned into the table captain they are removed from the table. I think this would be a great learning tool for us judges.

I actually like this idea. It can be somewhat anonymous in that you don't know who's scores were which judge, but it would be handy to see how you scored compared to others. If someone is usually scoring much lower than others (or much higher), then it would be a good way to point that out to them. I bet many of them don't even know that their scores are way off of what others are scoring it.

I also checked out FBA's requirements for judges and they require you to score at least once a year or you have to retake the judge class. I wouldn't mind KCBS doing something like that also.

BigBarry
08-26-2010, 02:31 PM
I'd send the proposal to the BoD.

Here is my email:

Greetings,

I am writing to request a motion be put forward at the next board meeting.

KCBS is the largest BBQ sanctioning body in the world. As such, they have developed a first class judging system.

However, every system can be improved and thus, I suggest the following process for your review.

Even though the lowest score at each table is discarded, if two judges scored much lower than the others at the table, that second-lowest score can critically hurt a team's chance to place in or win a category, GC or RGC. There should be an active feedback process for the judges to make sure that they are following the guidelines that they were originally introduced to during training. This process will include two new components:

1.) A review should be done of each judge's scoring when cards are turned in. The table captain can then provide active feedback to any judges that are not scoring within the table average. The feedback can be discrete or public (depending on how the KCBS feels it should be dealt with).

2.) Each scoring card should have the CBJ # and name printed on it. The CBJ # should be entered into the scoring system with their scores to track each CBJ. This data can then be sent to KCBS and analyzed (automatically - easy programming) against the other CBJs at the contest to see how far off each CBJ is from the norm at the contest. You can also see historical trending and provide feedback to those that trend lower via email. A weighting system can be developed to provide a realistic range of reasonable scoring. Non-CBJ scores should also be analyzed to see how effective the pre-contest training is (if any is provided). You can then also track the CBJ to non-CBJ ratios for a contest and see the judging disparities between each.

These suggestions are designed to improve the both the consistency of the CBJs and the quality of non-CBJs (via contest rep training onsite).

I thank you for your consideration.

Regards,

GREGORY BARRY 22347
PATRICIA BARRY22354 CBJ #:22354
TEAM NAME: BIG BARRY Q

PimpSmoke
08-26-2010, 03:31 PM
That looks like good suggestions to me. I'm wondering if other members following up with the BOD would help get some attention.

BigBarry
08-27-2010, 09:47 AM
That looks like good suggestions to me. I'm wondering if other members following up with the BOD would help get some attention.

Shot down in less than 1 hour:


Thank you for your suggestions.
Let me address that your first part is presently the policy of KCBS to talk to all judges who are plus or minus 2 pts from the mean at the table on their overall card. (not a single score). I hope that all Reps are doing their jobs. But that is often not easy to determine.

Second, we have the tracking software, but have not used it due to many factors. 1 is the absolute right of organizers to pick their judges and 2 to prevent black listing.

As to training of non cbj's there is approximately 1 hour to seat, sort, instruct, give the oath and be ready for judging. This prevents too much training of non cbj's. But we do not have a rule as to mandatory cbj's. We began to publish percentage of CBJ's on the contest web stite portion of the kcbs web site, to influence organizers to use more cbj's and that did seem to help.

Thank you for your thoughts.
Merl

BigBarry
08-27-2010, 09:53 AM
My response:

Merl,

As for the the tracking software, you are forgetting that the data would be confidential and held by the KCBS. An organizer would not be able to blacklist a judge because they would not have the performance metrics and analysis.

You would not be dictating whom the organizers can choose as judges, only providing feedback to the CBJs on their performance. Why not review/analyze (after the contest) their performance on all scores?

I understand that other sanctioning bodies more proactively monitor their judges and these practices are proven to work.

Without any form of CBJ evaluations, you are diluting the main reason for a contest to be sanctioned by the KCBS.

I would respecfully request that you present this to the board for review.

Regards,

Greg

mfreeman73
08-27-2010, 10:39 AM
I highly doubt they're going to change their system now and start keeping track of judges and talking to those that score low. But, I still think the idea up above about giving the judges a printout after the competition showing what they scored and what the other judges at their table scored would be very helpful.

Mainly because the judges that tend to score low probably don't realize that their scoring is so off from other judges. We don't get to see what other judges are writing down so there's no way to know. Sure we may talk about it after the cards are handed in, but no one discusses actual numbers. I know as a CBJ I would love to see how my numbers are comparing to other people's numbers. I certainly want to be fair to the competitors and if my scoring is way off from what other CBJ's score, I'd like to know.

So, my thinking is that the people we would label as bad judges are probably not intending to be bad, but rather, they don't even know they are bad. While competitors get feedback every competition, in the form of scores, there is no feedback for judges on whether they are doing their part also. Yes, I know the rep is supposed to step in if he sees numbers that are way off, but how often does this happen? I haven't seen it happen yet, although, I haven't done that many comps just yet.