Thoughts on judging

Do judges need more training?

  • Yes, I agree

    Votes: 38 67.9%

  • No, I do not agree

    Votes: 10 17.9%

  • No sure

    Votes: 4 7.1%

  • Who cares

    Votes: 4 7.1%
  • Total voters
    56
smoke-n-my-i's

smoke-n-my-i's

is Blowin Smoke!
Joined
Aug 27, 2007
Messages
1,633
Reaction score
128
Points
0
Location
Eustis, FL
The more I am around, here and at comps, the more I hear of rumblings of poor judging.

I too wonder if there should be more training, mentoring, requirements. It is hard to understand when you turn in the same meat in the same box, that you get say 4 9's than a 6 and a 5 or less say for taste..... WOW... where did that come from? IMHO, preference... nothing else.

Here is a prime example for appearance.... http://www.bbqcritic.com/17/post/2012/05/ribs-box-92-frostbuster-2012.html#comments

[FONT=&quot]Larry CBJ/KCBS/GBA [/FONT]
[FONT=&quot]05/08/2012 3:24pm[/FONT]
[FONT=&quot]7 - Gloppy sauce, uneven spacing, and clean the box. Attention to detail would surely get this box a 9.[/FONT]

[FONT=&quot]John - KCBS/MCBJ, CTC, Rep; MBN [/FONT]
[FONT=&quot]05/08/2012 5:16pm[/FONT]
[FONT=&quot]Easy 8 and perhaps a 9. Nice box. I ignored the smudge on the inside front of the box because that could have happened when the table captain unfastened the lid prior to presenting for appearance.[/FONT]
[FONT=&quot]BBQ Critic [/FONT]
[FONT=&quot]05/09/2012 4:51am[/FONT]
[FONT=&quot]John - Good call on ignoring that smudge inside the box. You are correct. The smudge could be the Table Captain's fault or even someone handling the box at the turn-in table. Congrats on giving the cook the benefit of the doubt.

-- BBQ Critic[/FONT]


Yes, I do a lot of surfing about the topic, and this is a prime example.... judges and table captains are instructed NOT to score down for sauce on the lid or box.... so, is this a lack of training, listening is class, or just personal preference????

I think this is just one example why so many say that the judges in general need more mentoring, training, etc....

These are just my thoughts only...

Comments are welcome, good or bad.
 
As a CBJ with a good number of contests judged, I still think that CONTINUAL education is a judge's best friend to provide the best possible judging for the cooks. Urging the judges to communicate with one another on their opinions of each category should be HIGHLY encouraged by the Table Captains.
In my humble opinion, the "rogue judge" (i.e., judge #6) is the one who thinks that he/she knows it all and won't listen to others or try to learn anything different from their own personal preferences. Their opinion of what is "good" is so limited that it is rare that they actually give high scores.

Just my $0.02.
 
I am seeing a trend back to judges only using the numbers 9, 8, and 7 for scoring entries instead of using the entire scoring system as taught. Part of this is due to comments being made by reps at judge's meetings like; teams spend a lot of money to be here to compete, give them the benefit of the doubt, give a good score if they did the best they could, and so on. Judges also have gone to using only these numbers so they will not be asked to fill out a comment card. I was a table captain at a recent contest where there were three 180 scores. Now, maybe those three meat entries were really that good, but at the table I was at no judge used any number on their card other than 9, 8, or 7 for every entry in all four meat categories except for two that had a chicken entry that was bloody raw and based on what I saw and sampled there were entries that should have not been anywhere near a 7 (above average). As much as cooks may want to think everything they turn in is great, judges have the responsibility to use the entire scale in scoring the entries they are judging. My point is that I agree that more training is needed to get judges to use he entire scale from 2 to 9 and not just the top three numbers. When you have four judges at a table using only 9, 8, 7 and you have two that are trying to use the entire scale from 2 to 9 then you end up with a scoring differential that does not look right.
 
bbq.tom said:
As a CBJ with a good number of contests judged, I still think that CONTINUAL education is a judge's best friend to provide the best possible judging for the cooks. Urging the judges to communicate with one another on their opinions of each category should be HIGHLY encouraged by the Table Captains.
In my humble opinion, the "rogue judge" (i.e., judge #6) is the one who thinks that he/she knows it all and won't listen to others or try to learn anything different from their own personal preferences. Their opinion of what is "good" is so limited that it is rare that they actually give high scores.

Just my $0.02.
Click to expand...

What you are talking about is more experience rather than education. Talking about the BBQ and discussing your opinions is a great idea as it adds to the judges experience about BBQ and how we all taste it differently.

Continuing education is more along the lines of going to class, seminar, webinar, etc that tries to teach a specific aspect of the BBQ or whatever.

I think what you are discussing is a great idea and should be happening, but continuing education could be a slippery slope of unintended consequences. Who decides what judges should consider "real competition BBQ"? Taste is too subjective. You will never please all of the people all of the time.
 
Taste is very subjective, but let's look at the example of the appearance score of Larry and John in my original post....

where did Larry get the idea to score down because of the sauce on the box????

John hit it right on target for what is taught in the classes.....
 
I think we need more time spent at the contest reminding judges of things like judge the entry as presented not just on what they like.

I also think there's no way to control personal preferences. Cooks should cook for the majority looking to get consistent scores. We can't control who judges, that's up to organizerss.

I also think we can't control the judges attitudes. They wake up on the wrong side of the bed, have a polite discussion with their spouse before heading to a contest and for that day their scores may drop. Next week they are right on with others. It's human nature and cooks have to live with it. Cooks also have bad days where they mess up. One rib has very little meat and the others are great and we say why did I get a low score. The cook may not even realize they messed up but the judges sure does. Even the best do things like this.

In the end, the cooks cook, the judges do the best they can that day and we move on to next week. I get as frustrated as everybody else when I get all 8's and 9's with a 766 but until we have paid professional judges that's life. I'm not advocating paid professional judges btw.
 
QN said:
I am seeing a trend back to judges only using the numbers 9, 8, and 7 for scoring entries instead of using the entire scoring system as taught. Part of this is due to comments being made by reps at judge's meetings like; teams spend a lot of money to be here to compete, give them the benefit of the doubt, give a good score if they did the best they could, and so on. Judges also have gone to using only these numbers so they will not be asked to fill out a comment card. I was a table captain at a recent contest where there were three 180 scores. Now, maybe those three meat entries were really that good, but at the table I was at no judge used any number on their card other than 9, 8, or 7 for every entry in all four meat categories except for two that had a chicken entry that was bloody raw and based on what I saw and sampled there were entries that should have not been anywhere near a 7 (above average). As much as cooks may want to think everything they turn in is great, judges have the responsibility to use the entire scale in scoring the entries they are judging. My point is that I agree that more training is needed to get judges to use he entire scale from 2 to 9 and not just the top three numbers. When you have four judges at a table using only 9, 8, 7 and you have two that are trying to use the entire scale from 2 to 9 then you end up with a scoring differential that does not look right.
Click to expand...

Yes, the last comp I was table captain at, I did hear this, but they are doing away with the comment cards (at least for now)... so why the
"push" for the higher scores????

I tend to glance at the scores just to make sure the score cards are filled out properly only, and not as to what each judge scores...

I think KCBS has implemented a system to track judges to see who are "no shows", scoring patterns, etc.... I think that is what I have read and/or seen... correct me if wrong please.

I tossed this back to the judging committee at KCBS as well... maybe they will continue to look at this more and get the "training" more up to speed with having the reps remind each judge not to look at the sauce on the lids,etc.... The reps at the last couple of comps I have been at have done this, so are the judges just not listening, or do they already have a mindset of what is right or wrong???? After reading those comments on my original post, I tend to believe that "Larry" has that mindset of what he expects.....
 
Mark

Had a discussion recently with a master FBA judge who also does MBN and KCBS. I asked what is average. Amazingly he said Sonnys was average. Another master judge said average based on their judging experience. I would expect nothing below 7 from the first and some lower from the second. If we just defined average as Sonnys then there would be a lot more consistency. That's also going to work for novice judges. Now maybe we lower what average is so that more numbers come into play. Now if I got average scores for my chicken, that means they thought it compared to Sonnys and I'd feel pretty bad about my chicken and be looking to improve. When we moved south I tried Sonnys chicken and I think every cook down here can and usually does better than that.
 
QN said:
Part of this is due to comments being made by reps at judge's meetings like; teams spend a lot of money to be here to compete, give them the benefit of the doubt, give a good score if they did the best they could, and so on. .
Click to expand...

I really hope this isn't the case, but can easily see it done at just about any contest. That is unfair to us as competitors. I do not deserve a 9 if it should be a 6. If i get a 9 on something, then i think he/she really meant to give that, and then i think my stuff is good. I can't learn with B.S. scores. I spend a lot of money to be judged fairly. (and others for that matter).. I can see me getting 11th in ribs, and another guy gettting 10th, because at his table there were two or so judges who feel sorry for teams who turned in crap. I want to be beat by better BBQ, not by judges who love to give out "Participant awards"
 
There needs to be some sort of standards put into place to give the judges some guidance. I'm now two contests into my experience level, so maybe my opinion doesn't count for much here, but I have noticed there is no consistency to judging.

A perfect example this weekend is our brisket. Not going to argue the taste or tenderness category since I thought it was a bit dry. However, it did look damn good, and five of the judges agreed with 9s. Then one judge gave us a 6!

Not saying any of them were deserved or not, but how can you have 5 judges give you a 9 and then one gives you a 6? An 7 or 8 I can see, but that's a big chasm between a 6 and 9.
 
Ford said:
Mark

Had a discussion recently with a master FBA judge who also does MBN and KCBS. I asked what is average. Amazingly he said Sonnys was average. Another master judge said average based on their judging experience. I would expect nothing below 7 from the first and some lower from the second. If we just defined average as Sonnys then there would be a lot more consistency. That's also going to work for novice judges. Now maybe we lower what average is so that more numbers come into play. Now if I got average scores for my chicken, that means they thought it compared to Sonnys and I'd feel pretty bad about my chicken and be looking to improve. When we moved south I tried Sonnys chicken and I think every cook down here can and usually does better than that.
Click to expand...

Not sure what Sonny's is. Have never had it. Guess that is a new rule that needs to be added for judge education. Must go eat Sonny's to see what average is... :becky:
RE: KCBS; old rule was to start at 9. then start at 6. now just score each entry on it's own with no starting point. I think a lot of judges have gone back to starting at 9.
 
Outnumbered said:
There needs to be some sort of standards put into place to give the judges some guidance. I'm now two contests into my experience level, so maybe my opinion doesn't count for much here, but I have noticed there is no consistency to judging.

A perfect example this weekend is our brisket. Not going to argue the taste or tenderness category since I thought it was a bit dry. However, it did look damn good, and five of the judges agreed with 9s. Then one judge gave us a 6!

Not saying any of them were deserved or not, but how can you have 5 judges give you a 9 and then one gives you a 6? An 7 or 8 I can see, but that's a big chasm between a 6 and 9.
Click to expand...

Maybe that one judge was trying to use the entire scoring scale and thought it looked "average"; 6 = average. And, maybe the other judges all started at 9 and stayed there. I have no way of knowing, but it is possible.
 
Sonny's is a BBQ chain in the southern states.... when I use to live there... average in my book... on most days.

Outnumbered said:
There needs to be some sort of standards put into place to give the judges some guidance. I'm now two contests into my experience level, so maybe my opinion doesn't count for much here, but I have noticed there is no consistency to judging.

A perfect example this weekend is our brisket. Not going to argue the taste or tenderness category since I thought it was a bit dry. However, it did look damn good, and five of the judges agreed with 9s. Then one judge gave us a 6!

Not saying any of them were deserved or not, but how can you have 5 judges give you a 9 and then one gives you a 6? An 7 or 8 I can see, but that's a big chasm between a 6 and 9.
Click to expand...

My point exactly..... just like the example I gave abt the sauce in/on the box.... they are NOT suppose to take that into consideration on the scoring of the MEAT....
 
QN said:
Maybe that one judge was trying to use the entire scoring scale and thought it looked "average"; 6 = average. And, maybe the other judges all started at 9 and stayed there. I have no way of knowing, but it is possible.
Click to expand...

Could be, but that's a hell of leap to justify. Maybe the one judge with the 6 looked at it and said it's all average and the other five said that looks excellent. In any case, isn't that an argument for establishing some sort of consistency? Is there a definition of average?
 
Outnumbered said:
Could be, but that's a hell of leap to justify. Maybe the one judge with the 6 looked at it and said it's all average and the other five said that looks excellent. In any case, isn't that an argument for establishing some sort of consistency? Is there a definition of average?
Click to expand...

I am not trying to justify anything, just giving an example of what mght have happened. To my knowledge,there is no specific definition of "average" other than that it is represented by a score of "6". I agree that it would be nice to have more consisten scoring. Back when everyone stated at 9 things were fairly consistent, but there were multiple180 scores at every contest because so many 9's were given consistently.
 
Outnumbered said:
There needs to be some sort of standards put into place to give the judges some guidance. I'm now two contests into my experience level, so maybe my opinion doesn't count for much here, but I have noticed there is no consistency to judging.
Click to expand...


2010 KCBS Judge’s Meeting – Mandatory Reading

This document or the audio copy of this document must be read or played for all judges at each KCBS Sanctioned contest. All Judges must attend this meeting and listen to the CD before they can judge. These Kansas City Barbeque Society judging procedures for 2010 are intended to provide the standards you should use in judging this contest. Remember, as a KCBS Judge, you are not judging by what you like but to the standards defined by KCBS. KCBS Sanctioning allows for blind judging only. Entries will be submitted in an approved KCBS numbered container provided by the Contest Organizer.

There already are standards in place, if you can find where they are actually listed. :frusty:
 
EatonHoggBBQ said:
There already are standards in place, if you can find where they are actually listed. :frusty:
Click to expand...

I took my class less than one month ago. There was discussion that the brisket should snap somewhat easily. But really that's about all that was discussed. No real standards we mentioned or discussed. A picture or something would be great to say, this here is average.

All I personally know about what is considered good BBQ meat is what that judge on BBQ Pitmasters said about bite-through skin, ribs that don't completely fall off the bone, and brisket that doesn't fall apart.

Ah hell, can I go back and change my vote to "doesn't matter?" Cause in the end, it really doesn't matter. You're going to get judged by that table and you get judged on what they like and what they don't. Everyone is going to have a different opinion on what is good.
 
QN said:
I am not trying to justify anything, just giving an example of what mght have happened. To my knowledge,there is no specific definition of "average" other than that it is represented by a score of "6". I agree that it would be nice to have more consisten scoring. Back when everyone stated at 9 things were fairly consistent, but there were multiple180 scores at every contest because so many 9's were given consistently.
Click to expand...

And honestly, it looked good. But for how good of a brisket it really was, the 6 was far closer to what it really was than 9 when you take how it tasted and the tenderness into account. :biggrin1:
 
You must log in or register to reply here.

Similar threads

L
Thoughts on Charcoal
2
Replies
23
Views
2K
Get’n Smokey
G
Hotch
Texas BBQ Brethren Bash Discussion Post.
Replies
13
Views
2K
frognot
frognot
cayenne
FIRST COOK on the new Offset Smoker!!! Pictures and comments...
2
Replies
22
Views
870
Bury Me In Smoke
B
LYU370
  • Poll
***Vote! BBQ Brethren "On a Roll" Throwdown - Vote!***
Replies
11
Views
2K
LYU370
LYU370
LYU370
  • Poll
*** Vote! BBQ Brethren August **SPECIAL** "On a Budget!" Throwdown Vote! ***
Replies
1
Views
708
Big George's BBQ
Big George's BBQ
Back
Top