Thoughts on judging

[smoke-n-my-i's]

The more I am around, here and at comps, the more I hear of rumblings of poor judging.

I too wonder if there should be more training, mentoring, requirements. It is hard to understand when you turn in the same meat in the same box, that you get say 4 9's than a 6 and a 5 or less say for taste..... WOW... where did that come from? IMHO, preference... nothing else.

Here is a prime example for appearance.... http://www.bbqcritic.com/17/post/201....html#comments

Larry CBJ/KCBS/GBA
05/08/2012 3:24pm
7 - Gloppy sauce, uneven spacing, and clean the box. Attention to detail would surely get this box a 9.

John - KCBS/MCBJ, CTC, Rep; MBN
05/08/2012 5:16pm
Easy 8 and perhaps a 9. Nice box. I ignored the smudge on the inside front of the box because that could have happened when the table captain unfastened the lid prior to presenting for appearance.
BBQ Critic
05/09/2012 4:51am
John - Good call on ignoring that smudge inside the box. You are correct. The smudge could be the Table Captain's fault or even someone handling the box at the turn-in table. Congrats on giving the cook the benefit of the doubt.

-- BBQ Critic


Yes, I do a lot of surfing about the topic, and this is a prime example.... judges and table captains are instructed NOT to score down for sauce on the lid or box.... so, is this a lack of training, listening is class, or just personal preference????

I think this is just one example why so many say that the judges in general need more mentoring, training, etc....

These are just my thoughts only...

Comments are welcome, good or bad.

[smoke-n-my-i's]

BTW, I have no clue who these judges are....

[bbq.tom]

As a CBJ with a good number of contests judged, I still think that CONTINUAL education is a judge's best friend to provide the best possible judging for the cooks. Urging the judges to communicate with one another on their opinions of each category should be HIGHLY encouraged by the Table Captains.
In my humble opinion, the "rogue judge" (i.e., judge #6) is the one who thinks that he/she knows it all and won't listen to others or try to learn anything different from their own personal preferences. Their opinion of what is "good" is so limited that it is rare that they actually give high scores.

Just my $0.02.

[QN]

I am seeing a trend back to judges only using the numbers 9, 8, and 7 for scoring entries instead of using the entire scoring system as taught. Part of this is due to comments being made by reps at judge's meetings like; teams spend a lot of money to be here to compete, give them the benefit of the doubt, give a good score if they did the best they could, and so on. Judges also have gone to using only these numbers so they will not be asked to fill out a comment card. I was a table captain at a recent contest where there were three 180 scores. Now, maybe those three meat entries were really that good, but at the table I was at no judge used any number on their card other than 9, 8, or 7 for every entry in all four meat categories except for two that had a chicken entry that was bloody raw and based on what I saw and sampled there were entries that should have not been anywhere near a 7 (above average). As much as cooks may want to think everything they turn in is great, judges have the responsibility to use the entire scale in scoring the entries they are judging. My point is that I agree that more training is needed to get judges to use he entire scale from 2 to 9 and not just the top three numbers. When you have four judges at a table using only 9, 8, 7 and you have two that are trying to use the entire scale from 2 to 9 then you end up with a scoring differential that does not look right.

[va92bronco]

Originally Posted by bbq.tom

As a CBJ with a good number of contests judged, I still think that CONTINUAL education is a judge's best friend to provide the best possible judging for the cooks. Urging the judges to communicate with one another on their opinions of each category should be HIGHLY encouraged by the Table Captains.
In my humble opinion, the "rogue judge" (i.e., judge #6) is the one who thinks that he/she knows it all and won't listen to others or try to learn anything different from their own personal preferences. Their opinion of what is "good" is so limited that it is rare that they actually give high scores.

Just my $0.02.

What you are talking about is more experience rather than education. Talking about the BBQ and discussing your opinions is a great idea as it adds to the judges experience about BBQ and how we all taste it differently.

Continuing education is more along the lines of going to class, seminar, webinar, etc that tries to teach a specific aspect of the BBQ or whatever.

I think what you are discussing is a great idea and should be happening, but continuing education could be a slippery slope of unintended consequences. Who decides what judges should consider "real competition BBQ"? Taste is too subjective. You will never please all of the people all of the time.

[smoke-n-my-i's]

Taste is very subjective, but let's look at the example of the appearance score of Larry and John in my original post....

where did Larry get the idea to score down because of the sauce on the box????

John hit it right on target for what is taught in the classes.....

[Ford]

I think we need more time spent at the contest reminding judges of things like judge the entry as presented not just on what they like.

I also think there's no way to control personal preferences. Cooks should cook for the majority looking to get consistent scores. We can't control who judges, that's up to organizerss.

I also think we can't control the judges attitudes. They wake up on the wrong side of the bed, have a polite discussion with their spouse before heading to a contest and for that day their scores may drop. Next week they are right on with others. It's human nature and cooks have to live with it. Cooks also have bad days where they mess up. One rib has very little meat and the others are great and we say why did I get a low score. The cook may not even realize they messed up but the judges sure does. Even the best do things like this.

In the end, the cooks cook, the judges do the best they can that day and we move on to next week. I get as frustrated as everybody else when I get all 8's and 9's with a 766 but until we have paid professional judges that's life. I'm not advocating paid professional judges btw.

[smoke-n-my-i's]

Originally Posted by QN

I am seeing a trend back to judges only using the numbers 9, 8, and 7 for scoring entries instead of using the entire scoring system as taught. Part of this is due to comments being made by reps at judge's meetings like; teams spend a lot of money to be here to compete, give them the benefit of the doubt, give a good score if they did the best they could, and so on. Judges also have gone to using only these numbers so they will not be asked to fill out a comment card. I was a table captain at a recent contest where there were three 180 scores. Now, maybe those three meat entries were really that good, but at the table I was at no judge used any number on their card other than 9, 8, or 7 for every entry in all four meat categories except for two that had a chicken entry that was bloody raw and based on what I saw and sampled there were entries that should have not been anywhere near a 7 (above average). As much as cooks may want to think everything they turn in is great, judges have the responsibility to use the entire scale in scoring the entries they are judging. My point is that I agree that more training is needed to get judges to use he entire scale from 2 to 9 and not just the top three numbers. When you have four judges at a table using only 9, 8, 7 and you have two that are trying to use the entire scale from 2 to 9 then you end up with a scoring differential that does not look right.

Yes, the last comp I was table captain at, I did hear this, but they are doing away with the comment cards (at least for now)... so why the
"push" for the higher scores????

I tend to glance at the scores just to make sure the score cards are filled out properly only, and not as to what each judge scores...

I think KCBS has implemented a system to track judges to see who are "no shows", scoring patterns, etc.... I think that is what I have read and/or seen... correct me if wrong please.

I tossed this back to the judging committee at KCBS as well... maybe they will continue to look at this more and get the "training" more up to speed with having the reps remind each judge not to look at the sauce on the lids,etc.... The reps at the last couple of comps I have been at have done this, so are the judges just not listening, or do they already have a mindset of what is right or wrong???? After reading those comments on my original post, I tend to believe that "Larry" has that mindset of what he expects.....

[Ford]

Mark

Had a discussion recently with a master FBA judge who also does MBN and KCBS. I asked what is average. Amazingly he said Sonnys was average. Another master judge said average based on their judging experience. I would expect nothing below 7 from the first and some lower from the second. If we just defined average as Sonnys then there would be a lot more consistency. That's also going to work for novice judges. Now maybe we lower what average is so that more numbers come into play. Now if I got average scores for my chicken, that means they thought it compared to Sonnys and I'd feel pretty bad about my chicken and be looking to improve. When we moved south I tried Sonnys chicken and I think every cook down here can and usually does better than that.

[BC Squared]

While I agree there may be poor judges out there, it sure doesn't seem to affect the consistent teams....so how big of an issue is it really?

[Podge]

Originally Posted by QN

Part of this is due to comments being made by reps at judge's meetings like; teams spend a lot of money to be here to compete, give them the benefit of the doubt, give a good score if they did the best they could, and so on. .

I really hope this isn't the case, but can easily see it done at just about any contest. That is unfair to us as competitors. I do not deserve a 9 if it should be a 6. If i get a 9 on something, then i think he/she really meant to give that, and then i think my stuff is good. I can't learn with B.S. scores. I spend a lot of money to be judged fairly. (and others for that matter).. I can see me getting 11th in ribs, and another guy gettting 10th, because at his table there were two or so judges who feel sorry for teams who turned in crap. I want to be beat by better BBQ, not by judges who love to give out "Participant awards"

[Outnumbered]

There needs to be some sort of standards put into place to give the judges some guidance. I'm now two contests into my experience level, so maybe my opinion doesn't count for much here, but I have noticed there is no consistency to judging.

A perfect example this weekend is our brisket. Not going to argue the taste or tenderness category since I thought it was a bit dry. However, it did look damn good, and five of the judges agreed with 9s. Then one judge gave us a 6!

Not saying any of them were deserved or not, but how can you have 5 judges give you a 9 and then one gives you a 6? An 7 or 8 I can see, but that's a big chasm between a 6 and 9.

[QN]

Originally Posted by Ford

Mark

Had a discussion recently with a master FBA judge who also does MBN and KCBS. I asked what is average. Amazingly he said Sonnys was average. Another master judge said average based on their judging experience. I would expect nothing below 7 from the first and some lower from the second. If we just defined average as Sonnys then there would be a lot more consistency. That's also going to work for novice judges. Now maybe we lower what average is so that more numbers come into play. Now if I got average scores for my chicken, that means they thought it compared to Sonnys and I'd feel pretty bad about my chicken and be looking to improve. When we moved south I tried Sonnys chicken and I think every cook down here can and usually does better than that.

Not sure what Sonny's is. Have never had it. Guess that is a new rule that needs to be added for judge education. Must go eat Sonny's to see what average is...
RE: KCBS; old rule was to start at 9. then start at 6. now just score each entry on it's own with no starting point. I think a lot of judges have gone back to starting at 9.

[QN]

Originally Posted by Outnumbered

There needs to be some sort of standards put into place to give the judges some guidance. I'm now two contests into my experience level, so maybe my opinion doesn't count for much here, but I have noticed there is no consistency to judging.

A perfect example this weekend is our brisket. Not going to argue the taste or tenderness category since I thought it was a bit dry. However, it did look damn good, and five of the judges agreed with 9s. Then one judge gave us a 6!

Not saying any of them were deserved or not, but how can you have 5 judges give you a 9 and then one gives you a 6? An 7 or 8 I can see, but that's a big chasm between a 6 and 9.

Maybe that one judge was trying to use the entire scoring scale and thought it looked "average"; 6 = average. And, maybe the other judges all started at 9 and stayed there. I have no way of knowing, but it is possible.

[smoke-n-my-i's]

Sonny's is a BBQ chain in the southern states.... when I use to live there... average in my book... on most days.

Originally Posted by Outnumbered

There needs to be some sort of standards put into place to give the judges some guidance. I'm now two contests into my experience level, so maybe my opinion doesn't count for much here, but I have noticed there is no consistency to judging.

A perfect example this weekend is our brisket. Not going to argue the taste or tenderness category since I thought it was a bit dry. However, it did look damn good, and five of the judges agreed with 9s. Then one judge gave us a 6!

Not saying any of them were deserved or not, but how can you have 5 judges give you a 9 and then one gives you a 6? An 7 or 8 I can see, but that's a big chasm between a 6 and 9.

My point exactly..... just like the example I gave abt the sauce in/on the box.... they are NOT suppose to take that into consideration on the scoring of the MEAT....