Judge accountability

A

Atlasman

Full Fledged Farker
Joined
Jun 12, 2012
Messages
222
Reaction score
80
Points
0
Location
Ny
Is there such a thing???.......or are they just let slide because of the excuse for incompetence (low score dismissed)


Example:

Our last chicken scores

988--888--999--999--857--988


I'm sorry but this should require an explanation. Further cause for concern is this judges AVG score he gave out all day for chicken was 27........3-5 points lower than every other judge at his/her table. So this person was at a table that EVERY other judge thought got good chicken all day yet they graded every entry they tasted poorly.

I wonder if these judges even comprehend a little bit what the consequences of their actions can be. To see teams get denied a chance at the Jack and have contests and championships effected by such silliness is a little disheartening when you factor in the amount of hard work, time and money that is invested by cooks and teams trying to have a successful season.
 
Assuming this was a KCBS contest....

1) Comment cards are currently suspended, so this judge could not provide an explanation to the team. Hopefully his Table Captain asked him for one when he turned in his slip.

2) Disparate scores are most common in chicken, where you are typically cooking separate pieces of meat. It is entirely possible for one thigh of six to deserve significantly lower scores than the other five.

3) Reps work to distribute boxes equally to all tables to minimize the damage of a low-scoring judge or table. This is why statistically a 24 team contest is the most fair, as each team's box should hit each table exactly one time. With more than four tables, luck becomes a big factor for teams that manage to miss the cold table.

If one judge was lowballing for the whole category, we would hope the Table Captain/Rep would have pointed this out and nudged them back to the centerline. I have seen this happen, usually with new judges and they usually take the correction well and are scoring back in range with the rest of the table.
 
I feel your pain- Ribs in Dover

989 -- 988 -- 856--765--878--989

Please fix the comment cards !!!!
 
Atlasman said:
Is there such a thing???.......or are they just let slide because of the excuse for incompetence (low score dismissed)


Example:

Our last chicken scores

988--888--999--999--857--988


I'm sorry but this should require an explanation. Further cause for concern is this judges AVG score he gave out all day for chicken was 27........3-5 points lower than every other judge at his/her table. So this person was at a table that EVERY other judge thought got good chicken all day yet they graded every entry they tasted poorly.

I wonder if these judges even comprehend a little bit what the consequences of their actions can be. To see teams get denied a chance at the Jack and have contests and championships effected by such silliness is a little disheartening when you factor in the amount of hard work, time and money that is invested by cooks and teams trying to have a successful season.
Click to expand...


lulz...you should have saved this for the silly season.

You got a bad score. It happens...it actually got dropped so it didnt impact your score. You probably didnt walk, but that score wasnt the reason.

Also it is entirely possible that piece of chicken earned that score.
 
MBN is tracking individual judge scores, and thoroughly reviewing its judges...and then booting or retraining the problem ones. Wish KCBS would jump on the Bandwagon.

And before a Judge flames me, I am a KCBS, MIM, and MBN CBJ. The vast majority of judges are good, just, and fair...but you know as well as I there are odd ducks that score on wacky things like "I dont like muffin pan chicken so I never give it a 9" or "I score down for not have pulled pork in a box" - both statements were made by CBJ's at contests.

When I screw up as a cook, a judge gives me a bad score...when a judge screws up or is a bad judge, he or she ruins a team's chances of winning and costs us cooks a lot of money...HUGE difference

So yeah, judge accountability is an issue.
 
I see a lot of judges just using the numbers 9-8-7 for every entry they score at KCBS events. I have even heard judges say that they just can't give anything lower than 7 even if it should be 5 (below average). If you have someone trying to use the entire scale using all the numbers from 2 to 9 and other judges at the same table are using only 9-8-7 then the judge trying to score correctly looks like a low scoring judge. There was even an article in the Bullsheet recently about this.
From the article in the September Bullsheet;
Another habit I see in judging is the 7,8,9 judges. The numbers available
to us as judges are 2 thru 9.
Granted 2,3,4 are seldom seen at one of my contests but they do happen
and I for one have no problem writing them on my card. If a sample
is that bad usually I’m not the only one at the table with that score. But I
have also heard a judge say “I never give anything less than a 7 because
I feel bad for the cook”, That’s nice and all but what about the cook? A 7
means above average, are you doing the cook a disservice by telling him
his entry is above average at this contest and the next contest judges give
him a realistic score of 5 and now he is extremely confused? Wouldn’t it
be more fair to truthfully give him the score that really reflects his entry so​

that he knows where he stands and can begin the process of improving?
 
Atlasman and to an extent Rolfe, I think the main points to come away with here have been listed above, being:

> Disparate scores are most common in chicken, where you are typically cooking separate pieces of meat. It is entirely possible for one thigh of six to deserve significantly lower scores than the other five.

Ribs being second most common because of different ends, different racks, etc...


Also, scores varying 2 numbers are normal and to be accepted. Frankly, it's very subjective (other than tenderness). Even then, tenderness does have a tendency to affect taste scores. I think that's what we're seeing on Rolfe's scores above.

The 5's and 6's to the 9's displayed in Rolfe's example is noteworthy IMHO, but also much more easily explained in ribs or chicken. The variance is pretty huge, but I've seen it in my own Q a time or two. One time we put forth ribs from 2 different racks, only to find out that one of the racks (had to be the un-sampled one, didnt it, da**ed laws of nature) was skunky. We were slammed in taste on those 2 ribs; rightfully so...

I'm not saying yours were skunky. However, it appears as though you had 2 or 3 ribs there that probably were tough as nails (if I had to guess).
 
I can understand differences in taste and tenderness. Especially in ribs and chicken. However, appearance should be pretty consistent. How can someone come up with a 7 when the rest are 9's and 8's?
 
I actually competed in an event earlier this summer and at the cook's meeting we were told that all judges had erasers and that they could change our scores based on what other entries scored. When asked why they didn't use a set standard for each category instead of comparative judging, which gives us no mark to strive for, they did not offer an answer except to explain it again. Seems to me that any event using comparative scoring hurts its cooks simply because the cooks not longer have an idea of what T T and A should look like. That's like saying if you have more tackles and bigger players than the other team, you should always win the game.
 
nkay said:
I actually competed in an event earlier this summer and at the cook's meeting we were told that all judges had erasers and that they could change our scores based on what other entries scored. When asked why they didn't use a set standard for each category instead of comparative judging, which gives us no mark to strive for, they did not offer an answer except to explain it again. Seems to me that any event using comparative scoring hurts its cooks simply because the cooks not longer have an idea of what T T and A should look like. That's like saying if you have more tackles and bigger players than the other team, you should always win the game.
Click to expand...

What sanctioning body?

Example...Each box in Blind Judging in KCBS is judged on its own merits. In MBN entries are judged agianst each other...best on table wins.
 
Pole D said:
I can understand differences in taste and tenderness. Especially in ribs and chicken. However, appearance should be pretty consistent. How can someone come up with a 7 when the rest are 9's and 8's?
Click to expand...
As Lakedogs stated above, "scores varying 2 numbers are normal and to be accepted". 7 is defined as above average, so getting a 7 along with 8's and 9's is not inconsistent and the lowest score is dropped.
 
Pole D said:
I can understand differences in taste and tenderness. Especially in ribs and chicken. However, appearance should be pretty consistent. How can someone come up with a 7 when the rest are 9's and 8's?
Click to expand...

In general I agree with you, BUT . . . darned if the devil isn't in the actual details...

First, I hear multiple times from many people that appearance score is the most objective (and least subjective) of them. I happen to disagree 100%. Frankly, what really turns your taste buds on (in appearance) may not set my lights afire and visa versa. Also, imagery comes in to it for some, and I'll author a point that if a piece of something comes across your table at a restaurant and it looks completely like a turd, something tells me that you're not thinking: "DAMN, I JUST MUST EAT THAT TURD". As it happens, plenty MM's that aren't displayed well look like turds. Pulled pork can be made to look like cat food if you aren't careful. I've seen chicken that looked like potatoes, and many others that didn't look like food at all. It's tough to say "Damn, I want to eat those taters" when we're trying to judge how appetizing the chicken entry...

So, honestly, I think a 2 point variance (in a KCBS system) is normal. However, 3 or 4 point variance, unless something is very off, should be questioned.
 
Is it fair to say that geography/where comp is held has something to do with the outcome judging?
 
CivilWarBBQ said:
Assuming this was a KCBS contest....


If one judge was lowballing for the whole category, we would hope the Table Captain/Rep would have pointed this out and nudged them back to the centerline. I have seen this happen, usually with new judges and they usually take the correction well and are scoring back in range with the rest of the table.
Click to expand...

I hope they are doing this after the contest and not between turn-ins.
Someone correct me but I dont think a judge should be influanced in the middle of a contest. ????
 
I started out as a judge and always looked at things through a judges eyes, then when I started cooking i looked at things through a competition cooks eyes. As time has progressed I have tried to see things with OPEN eyes.

Atlasman I will only address your chicken score and this is only a possible explanation.

Appearance: four 9s and two 8s= pretty consistent
Taste:two 9s, three 8s and a 5= maybe one judge got a bite with way more rub or pepper or spicier than expected or whatever but it still may have been unintentionally different.
Tenderness:three 8s two 9s and a 7= you want to have more 9s than 8s but not bad. Maybe the one judge got a rubbery skin, maybe skin came off completely.

You cant tell for sure but by having more 8s than 9s in taste and tenderness you can tell there may be minor issues. All in all not bad but I hope I pointed out there could be very legitimate reasons for your score.

And if you dont like any of those explanations, then use the explanation a buddy of mine uses every time he gets a score he doesnt like. " Why is that person a judge, he didnt come here wanting BBQ he was wanting Chinese food"
 
I am a KCBS CBJ and I do backyard competitions. Judging is not the easiest job in the world. I enjoy it, and I am as fair as I can be, just like I expect the judges to be when I turn in ribs or chicken or whatever in a competition. I re-read the red KCBS judge's manual the night before I judge at a competition to make sure I am familiar with the standards KCBS has set. If I judge meat that really tastes "average", then that is the score they are going to get. If a box looks excellent, then that is the score it is going to get for appearance. Low scores happen once in a while... this is a competition - someone has to win, and someone has to lose. As long as human beings are judging, it isn't going to be perfect, no matter how hard we judges try. And the lowest score is dropped... so unless Atlasman was tied with someone and it cost him GC or money or a call, is this really that big of a deal?
 
ITBFQ said:
I am a KCBS CBJ and I do backyard competitions. Judging is not the easiest job in the world. I enjoy it, and I am as fair as I can be, just like I expect the judges to be when I turn in ribs or chicken or whatever in a competition. I re-read the red KCBS judge's manual the night before I judge at a competition to make sure I am familiar with the standards KCBS has set. If I judge meat that really tastes "average", then that is the score they are going to get. If a box looks excellent, then that is the score it is going to get for appearance. Low scores happen once in a while... this is a competition - someone has to win, and someone has to lose. As long as human beings are judging, it isn't going to be perfect, no matter how hard we judges try. And the lowest score is dropped... so unless Atlasman was tied with someone and it cost him GC or money or a call, is this really that big of a deal?
Click to expand...

Yup it is.

YOU ARE A GOOD JUDGE. You do things right and true. I appreciate that more than you know. We cooks need EVERY Judge to be like you

BUT, you know there are judges out there that do things wrong, don't follow the rules, and get away with it because there is absolutely no accountability system in place, and that needs to be corrected. MBN is currently doing that, why can't KCBS?

Good judges would skate through with flying colors. They would catch the judges that need to be retrained on rules, the ones confused on what should be or shouldn't be in a box, and ESPECIALLY the few that don't care what the rules say...they do what they want.
 
You must log in or register to reply here.

Similar threads

S
Judging Question
Replies
16
Views
1K
Smokin D@wg@77
S
SweetHeatBBQnSC
Pron from this past weekends cookoff
Replies
9
Views
595
Big George's BBQ
Big George's BBQ
P
Diagnostic Scoring Idea
Replies
11
Views
573
masque13
masque13
F
KCBS Judging
2 3
Replies
52
Views
4K
tduffy
tduffy
O
Dissecting a Master Judge 'Cook with a Team' Requirement
Replies
7
Views
967
backdraftbbq
backdraftbbq
Back
Top