Is the KCBS BOD schizophrenic?

[billygbob]

From the Quick Notes from the Board 10-13-10 on KCBS web site:

Rules Committee - Candy Weaver

"...due to a number of different factors, KCBS is evolving to a 3 or 4 number scoring system (i.e., 6-7-8-9), which leads to higher scores and more 180s..."

New Ideas Committee - Merl Whitebook

"...when it appears that a CBJ statistically is inconsistent in scoring (+/- 2 from the mean of the overall contest results,) at two or more contest, in a 12 month period, the CBJ will be mentored by the CBJ Chairperson. ... The motion was seconded by Ed Roith."

So here is one BOD member stating that judges don't use all the numbers and another BOD member motioning to effectively penalize members that do use all the numbers (seconded by the CBJ Committee - Ed Roith: No Report member).

Dollar-to-your-dime that that judging will become a 7-8-9 score contest because you are penalized for actually giving the correct score. In nine years as a CBJ and six as a CTC I know as a fact that many judges do not give less than a 6 because "that's low enough they won't win" and too many that won't give less than a 7 or 8 because "the cooks work hard and spend a lot of money". When I score my 3 or 4 for taste and tenderness on a burnt, dry piece of chicken and everyone else gives a 6 or 7 or 8 I'm the one that is considered the "outlier" and need a talking to - or banned.

<RANT>
I think drbbq on The BBQ Forum has it nailed: "This sounds like my 7yo girls soccer games. Give everyone 9's and a ribbon... and a box of juice".

If the BOD wants to track something, check for the judges that never score out of the same two numbers. I guarantee that over five contests (120 entries give or take - 5 contests x 4 meats x 6 entries) that if that judge never got a piece of crap, or something he'd like to have more of, then you've identified the real problem.

The real issue is that some judges are not scoring "honestly", either out of best intent or attitude. That doesn't eliminate the fact every judge has a different perspective, or the fact every piece of meat is not identical; there will be differences in scores. But if every judge gave an honest score then much of the outcry could be addressed.
</RANT>

[ThomEmery]

If a CBJ consistently scores well outside the norm of his tables
Maybe "honest" is not what he is

[Elegant Bear]

I am not a competitor. I took a CBJ class recently and the person who was supposed to cook for the class had a last minute personal emergency and replacement cooks were recruited from throughout the community. The replacements did not have to be competition cooks. Every piece of meat that I tasted that day, I would have sent back to the kitchen had it come from a restaurant. It was the worst BBQ I had tasted in my life. The class leader acknlowledged the problem and said that it would be valuable to us to know what the bad stuff really is. The scores on each sample were in the full range just at my table. There were 9's and 2's.

If there is a scoring problem, it is in the training of the new judges because there are more than 70 CBJ's that came out of my class that have not yet seen, felt or tasted a 7, 8 or 9. Further, from the comments above, some judges have their own "sub criteria" for scoring that are not represented in the KCBS rules.

[billygbob]

Originally Posted by ThomEmery

If a CBJ consistently scores well outside the norm of his tables
Maybe "honest" is not what he is

The primary point I'm trying to make is judges not scoring honestly is the issue, and that it is a not an uncommon occurrence; they skew the "norm". And I'll add that the proposed evaluation period of two contests is ludicrous from a statistical standpoint.

[SaucyWench]

Thom, I judged today, and there were swings in the scoring. 4 of us got sauced fat in the brisket box, 2 got nice crispy chunks. One rib box had 5 very nice ribs, yet the judge sitting beside me got a rib with barely enough meat to taste. His score was significantly lower than the rest of ours. Theoretically, with the tracking that should have started today, if he scores anything lower than the rest of the table within the next 12 months, he will be subject to evaluation. Should this judge be penalized for getting the one bad piece of chicken in a box on top of a rib with no meat?

[Bunny]

Originally Posted by SaucyWench

Thom, I judged today, and there were swings in the scoring. 4 of us got sauced fat in the brisket box, 2 got nice crispy chunks. One rib box had 5 very nice ribs, yet the judge sitting beside me got a rib with barely enough meat to taste. His score was significantly lower than the rest of ours. Theoretically, with the tracking that should have started today, if he scores anything lower than the rest of the table within the next 12 months, he will be subject to evaluation. Should this judge be penalized for getting the one bad piece of chicken in a box on top of a rib with no meat?

And that happens a lot more than you know. If a judge gets a really bad rib compared to the others, why are they out of the norm? They are talked to if they are out of the same realm of the other judges, but if they have a legitamate reason, so there lies a true score. We are not out there to sway judges opinions, just making sure they understand our procedures.

[Skinny Cook]

I see judge number ident theft comming ?

I judged the same contest as SaucyWench. I too had one rib that was not fit to feed to a dog. Couldn't have cut it off the bone with a razor blade. They got a low score compared to the rest of the box. The ribs in the box obviously came from several different slabs as the other judges at my table didn't have a like rib. The others were also a different color.
Sure not my inconsistancy as a judge that caused that.

[ThomEmery]

Judges will not be penalized
Education is not a penalty

The Rib example given would be noted by the Rep
If this same CBJ were given a bad entry at another contest
it would be noted once more
There are times when one judge will get a substandard piece
and perhaps it will be discovered that 2 times outside table norm
is too frequent the action step

[SaucyWench]

The problem I see with computerized judge tracking (which by the way, failed this week due to programing issues) is, how do we prove that a judge has been asked why a score, be it low or high in comparison with others at the table, is not within the norm? If a rep says it's a valid reason, will that override the tracking program? Since computers cannot know our reasons for scoring as we do, I fully expect to get a talking to next year, because I will not give a 6, 7 or 8 to something that deserves a 5 or less.

[Spydermike72]

Originally Posted by SaucyWench

The problem I see with computerized judge tracking (which by the way, failed this week due to programing issues) is, how do we prove that a judge has been asked why a score, be it low or high in comparison with others at the table, is not within the norm? If a rep says it's a valid reason, will that override the tracking program? Since computers cannot know our reasons for scoring as we do, I fully expect to get a talking to next year, because I will not give a 6, 7 or 8 to something that deserves a 5 or less.

I would hope the program would have a way to note a Reps comments. I dont know that for sure but I am taking a wild guess here...

[Smoke'n Ice]

At a contest this weekend the following scores on ribs were posted for the top 3 finishers:

1 34.8572 23.4286 30.2856 36.0000 35.4286 34.8572
2 34.8572 24.0000 33.7142 30.2856 33.7142 36.0000
3 35.8572 25.1428 35.4286 32.0000 29.7144 32.0000

I have looked at all of the results and suspect I can pick out the additional entries judged by these judges based on the score pattern. The question is, "Which Judge Should Be Re-educated -- Judge 1 for consistently having no variation or judge 2 for consistently being well below the rest of the table or did judge 2 perchance get the one bad rib in each of the boxes he judged? It might be that he needs new glasses

[billygbob]

Originally Posted by ThomEmery

Judges will not be penalized
Education is not a penalty

Further in the same motion "...Should the problems continue then the matter shall be brought to the Board for removal or further action by the board. ..."

Removal or "further action" is not a penalty?

[ThomEmery]

Implementing the nuts and bolts of this will take time
Yes there are more questions that need to be answered here
I can only comment on the concept
Maybe we can get a current BoD member to address these points

Thanks for taking the time

BoD action to remove a CBJ is at the last resort I would hope

[SaucyWench]

Originally Posted by Smoke'n Ice

At a contest this weekend the following scores on ribs were posted for the top 3 finishers:

1 34.8572 23.4286 30.2856 36.0000 35.4286 34.8572
2 34.8572 24.0000 33.7142 30.2856 33.7142 36.0000
3 35.8572 25.1428 35.4286 32.0000 29.7144 32.0000

I have looked at all of the results and suspect I can pick out the additional entries judged by these judges based on the score pattern. The question is, "Which Judge Should Be Re-educated -- Judge 1 for consistently having no variation or judge 2 for consistently being well below the rest of the table or did judge 2 perchance get the one bad rib in each of the boxes he judged? It might be that he needs new glasses

There's that dang judge #2 again!!!

Seriously though, there is little chance that the top 3 were at the same table, and according to the "2 points +/- from the mean" plan, judge #2 is not the only one out of line-some of the high scorers are too.

[Smoke'n Ice]

Originally Posted by SaucyWench

Seriously though, there is little chance that the top 3 were at the same table

I would not be willing to bet on this