KCBS BoD's Meeting Re: Judges Scores

[Okie Sawbones]

Start a new CBJ training VIDEO. All existing CBJs have to watch it and can watch it for free on kcbs.us when they login with their member number. Standardize the training, don't allow human instructor drift.

This is well and good, but it does nothing for tasting the meats and identifying the flaws. What is creosote? What is mushy/overcooked? Just what is too much tug? etc. Unfortunately, there is still a lot of subjectivity. What is too hot and spicy for one judge might be just right for me. You can't teach that.

 

[DUBBAGA]

It is hard to define what the "problem" is. I will say, starting when judge's scores were first recorded, the threat to send high/low scoring judges for retraining resulted in higher scoring (at least in CA). When table numbers began showing up on team score sheets, scores took another jump. I know of one instance when a judge encouraged others to "let's not be the TOD".

Retraining will never work. Until KCBS begins testing judges on rules and actual tasting (similar to what CA State does the wine judges), you will have judges all over the place. Worse yet, you have judges afraid to give the scores deserved, with 7 being their bottom number.

What is the purpose of the judge score tracking system if KCBS is not going to use it to make tables more equal. Why give teams all that information in the first place? All it does is cause us to bitch and moan about low scoring judges/tables.

I've looked at my scores on KCBS, and how the table voted, and I am usually less than 1 point difference. Example: My scores from March of this year, Brisket 33.25, table average 33.17, Chicken 33.48, table 33.62, Pork 31.88, table 31.86, Ribs 34.05, table 33.44. This doesn't seem to be of much use to KCBS for trying to place judges at various tables.

I've proposed a fix, call it a band aid or lipstick, to use the information readily available to help with a situation many think is broken. If the system is not going to be used to make tables more level, the simple thing is to do away with collecting the information. Then we can all go back to being in the dark about whether we hit a high/low scoring table.

I know cooks are convinced there are TODs, but honestly I have never seen it. I have been at a table where the rep shook his head and said, "The scores are pretty low here folks," but what are you to do when the entries are really poor? Table scores were 26-28 for ribs.

I'm not long for competing, maybe 10 more contests through the end of 2018. But I hope to see something done to produce a more balanced table assignment. It worked perfectly at the "King of the Smokers" where those of us who judged submitted our averages to help create balanced tables. I have a feeling the teams who were there appreciated that as well. But that was only 24 people. Too bad it will not happen again if this proposed rule/advisory gets approved. KCBS can do it and keep all that information anonymous. All they have to do is want to.

I'll leave it at that and go crawl back in my hole.

Okey Dokey Okie, you wrote:

I've looked at my scores on KCBS, and how the table voted, and I am usually less than 1 point difference. Example: My scores from March of this year, Brisket 33.25, table average 33.17, Chicken 33.48, table 33.62, Pork 31.88, table 31.86, Ribs 34.05, table 33.44. This doesn't seem to be of much use to KCBS for trying to place judges at various tables.

Based on the weighted scoring system, even if you gave a 9 8 9, your score would be 33.7028. As a result, I'm left scratching my head about how there is this uncontrollable phenomenon of over-sauced sweet ribs where you can't taste the meat, and as such, are regularly giving 7's and below. Your March 2017 average scores do not seem consistent with this proclamation... so, is it a real issue, or do you just have some free time between canning sessions

 

[Okie Sawbones]

Believe me, I agree!!!

My understanding is that the KCBS BOD is working on a 3-strike program that I believe will de-CBJ any judge who is a no-show three times. I might be off-base on this, but time will tell.

Meanwhile, there is a rumor floating around that a list might exist that has no-show and trouble-making judges. If it exists, I just hope that I'm not on it!!!

I'd like to see this myself. Common courtesy is not so common anymore. One judging that I did in OK this year has a solution -- they stated right in the letter to the judges that if you no show, you be NOT be considered the following year. I'd like to know how many no showed.

 

[ModelMaker]

I quit competing in KCBS contests in 2012. That was the same year I received a 5 and 9....on the same meat, at the same table and not a single comment card was issued.

With that kind of inconsistency with scoring, it was not much more than a crap shoot, so why bother?

No chance in hell that the 5 might have been the cookers fault?
Every piece of meat in that box was EXACTLY equal to the rest?
Did you find a judge that scored correctly and the other five were high scorers?
A certain spice or flavor that judge really dislikes (perfectly fair on their part)?

Sounds like "I wasn't treated exactly as I felt I should have so I'm not going to play anymore".
Your prerogative I guess, just don't damn the judge without knowing all the facts.
Not giving you a comment card? Judges prerogative, not his job to teach you competition BBQ cooking.
Ed

 

[Okie Sawbones]

Okey Dokey Okie, you wrote:

I've looked at my scores on KCBS, and how the table voted, and I am usually less than 1 point difference. Example: My scores from March of this year, Brisket 33.25, table average 33.17, Chicken 33.48, table 33.62, Pork 31.88, table 31.86, Ribs 34.05, table 33.44. This doesn't seem to be of much use to KCBS for trying to place judges at various tables.

Based on the weighted scoring system, even if you gave a 9 8 9, your score would be 33.7028. As a result, I'm left scratching my head about how there is this uncontrollable phenomenon of over-sauced sweet ribs where you can't taste the meat, and as such, are regularly giving 7's and below. Your March 2017 average scores do not seem consistent with this proclamation... so, is it a real issue, or do you just have some free time between canning sessions

Damn, you caught me. I try everything I can to be fair to cooks. I'll even push aside the sauce on over-sauced entries to try and taste the meat and rub. If appearance is a 9, tenderness is a 9, and I can get some taste under the goo, then they get an 8. I should give a 7, but I don't too often. Why? Score inflation. And reps who want high scores. High scores = more cooks. More cooks = successful event. It is a vicious cycle that feeds on itself. It doesn't mean that over saucing is not a problem. BTW, at a contest in May, my average rib score was 27.08, table was 26.6.

If you have a solution to fair scoring, I'm all ears. 6 is average BBQ. 60% off what I taste at a contest is average or below average. If 60% of my taste scores were a 6, I'd have people giving me a proctologic exam. :shocked:

 

[DUBBAGA]

I try everything I can to be fair to cooks. I'll even push aside the sauce on over-sauced entries to try and taste the meat and rub.

I can't imagine any cooks are going to be pleased to hear that you, as a CBJ, are messing with entries (for whatever reason), before you taste them. Judge what is presented to you, how it is presented to you :doh:

If appearance is a 9, tenderness is a 9, and I can get some taste under the goo, then they get an 8. I should give a 7, but I don't too often. Why? Score inflation. And reps who want high scores.

So you are part of the problem which you lament about. If you can't judge honestly and fairly to your own standards, then why are you continuing to be a judge? :doh::doh:

If you have a solution to fair scoring, I'm all ears.

How about each judge assigns whatever score they personally feel to be appropriate, for whatever reason. Be prepared to have the bizzals to stand behind their score, and justify it where appropriate. Further, follow the #5 of the KCBS Judges' Code of Conduct where it says

"I will be true to my own taste and will not attempt to impose my personal taste preferences on other judges"... don't try to start a movement, don't try to right a self-created injustice, that's not your function as a judge! :doh::doh::doh:

 

[Okie Sawbones]

I can't imagine any cooks are going to be pleased to hear that you, as a CBJ, are messing with entries (for whatever reason), before you taste them. Judge what is presented to you, how it is presented to you :doh:

There you go again, assuming facts not listed. I think you just want to try and bust my chops. I taste every entry AS PRESENTED, and if there is hope, I check under the gorilla goo. Fail.

So you are part of the problem which you lament about. If you can't judge honestly and fairly to your own standards, then why are you continuing to be a judge? :doh::doh:

:blah::blah::blah::blah::blah: Did you bother to check my March scores? All 6s and 7s for taste. Fail.

How about each judge assigns whatever score they personally feel to be appropriate, for whatever reason. Be prepared to have the bizzals to stand behind their score, and justify it where appropriate. Further, follow the #5 of the KCBS Judges' Code of Conduct where it says

"I will be true to my own taste and will not attempt to impose my personal taste preferences on other judges"... don't try to start a movement, don't try to right a self-created injustice, that's not your function as a judge! :doh::doh::doh:

As part of KCBS, I am free to speak my opinion, and relate the opinion of other judges. If you don't like it, then discuss it, instead of trying to bully me, or make childish, snide remarks. Fail. Seems like you have 3 strikes. Back to the dugout with you. BTW, I don't see any solutions from you, only vitriol.

:doh::doh::doh::doh::doh::doh::doh::doh::doh:

 

[DUBBAGA]

... and this is why we can't have nice things ...

KCBS weighting for 979 = 31.4056
KCBS weighting for 969 = 29.1084

Did you bother to check my March scores? All 6s and 7s for taste. Fail.

Here are the self-professed scores for March... I'll let everyone else draw their own conclusions about Okie's mastery of the facts

My scores from March of this year, Brisket 33.25, table average 33.17, Chicken 33.48, table 33.62, Pork 31.88, table 31.86, Ribs 34.05, table 33.44.

 

[Okie Sawbones]

... and this is why we can't have nice things ...

KCBS weighting for 979 = 31.4056
KCBS weighting for 969 = 29.1084

Did you bother to check my March scores? All 6s and 7s for taste. Fail.

Here are the self-professed scores for March... I'll let everyone else draw their own conclusions about Okie's mastery of the facts

My scores from March of this year, Brisket 33.25, table average 33.17, Chicken 33.48, table 33.62, Pork 31.88, table 31.86, Ribs 34.05, table 33.44.

I mistyped. I meant May scores. Brisket 29.36, table 30.5, chicken 33.2, table 31.8, pork 32.1, table 31.4, ribs 27.8, table 26.6.

 

[rorkin]

You cannot average random events and produce meaningful data. Seating judges by average score is as effective as seating them by shoe size. Brad hit it right on the head. It's a standards of education problem coupled with a lack of enforced policies. Everyone on the same page should be the goal; not 8's across the board.

KCBS needs to get on top of this quickly and exert some authority as another more serious issue is morphing out of this problem. Some Reps and volunteers now seem to be taking it upon themselves to try and fix what they consider inequities in the judging tent by ignoring standard KCBS procedure and implementing their own personal "fix". This type of freelancing at contests is a huge concern. Even if a KCBS procedure is flawed it's a KNOWN process. Contests are not laboratories for people to experiment using the cook teams as lab rats because they feel something is unfair. All contests should be run the same - or as close as humanly possible. If a cook team adds red tipped lettuce or apple slices in their box because they think it "looks better" they get DQ's because it's a hard and fast rules violation (black letter law). Judges judge samples in the order they are placed on their judge's mat. (black letter law). Why are people being allowed to modify the turn in procedure? This needs to be addressed ASAP. KCBS needs to issue an edict that judging procedures will be followed to the letter and, if they aren't, there will be consequences.

Exactly right..

 

[Fat Freddy]

No chance in hell that the 5 might have been the cookers fault?
Every piece of meat in that box was EXACTLY equal to the rest?
Did you find a judge that scored correctly and the other five were high scorers?
A certain spice or flavor that judge really dislikes (perfectly fair on their part)?

Sounds like "I wasn't treated exactly as I felt I should have so I'm not going to play anymore".
Your prerogative I guess, just don't damn the judge without knowing all the facts.
Not giving you a comment card? Judges prerogative, not his job to teach you competition BBQ cooking.
Ed

Ed, Since I havent done any contests this year. Things at the particular contest I will mention might have been different, I have not heard anything but 2016 Council Bluffs hit several arguments in this particular thread and your comment as well and points out a BIG issue to me. There was a celebrity judge(different argument for a different day) that the primary sponsor wanted to have there. He was giving 5's left and right. Top 3 overall actually got 6's from this judge where as the rest of the table was given 9's and 8's from the other judges at the table for a particular category(Top 3 Porky Butts, Smokey D's, Oversize Load) Now I would argue that 2 of those 3 would know if they should have gotten 6's. The other gets lucky once in a while.....Anyways this judge also was giving out 3's and 4's. i know one team that was so frustrated that they quit period, another team that was so frustrated that they talked about quitting but they did Not and now are having a great year. I talked to one of the reps and this judge was talked to but because was sponsor nothing could or was done.

So my point is sometimes judges are the issue, i wont throw all judges under the table. It is not always the cooks. And while you didnt say it, another post said they have never seen a TOD.....I guarantee this was a TOD, fortunately for at least the top 3 this judges scores were thrown out but again a good friend of ours Woodward got 555 from this judge on I believe ribs might have been chicken and 9's and 8's from the other judges. THAT WAS the judge not the cook.

 

[Okie Sawbones]

Ed, Since I havent done any contests this year. Things at the particular contest I will mention might have been different, I have not heard anything but 2016 Council Bluffs hit several arguments in this particular thread and your comment as well and points out a BIG issue to me. There was a celebrity judge(different argument for a different day) that the primary sponsor wanted to have there. He was giving 5's left and right. Top 3 overall actually got 6's from this judge where as the rest of the table was given 9's and 8's from the other judges at the table for a particular category(Top 3 Porky Butts, Smokey D's, Oversize Load) Now I would argue that 2 of those 3 would know if they should have gotten 6's. The other gets lucky once in a while.....Anyways this judge also was giving out 3's and 4's. i know one team that was so frustrated that they quit period, another team that was so frustrated that they talked about quitting but they did Not and now are having a great year. I talked to one of the reps and this judge was talked to but because was sponsor nothing could or was done.

So my point is sometimes judges are the issue, i wont throw all judges under the table. It is not always the cooks. And while you didnt say it, another post said they have never seen a TOD.....I guarantee this was a TOD, fortunately for at least the top 3 this judges scores were thrown out but again a good friend of ours Woodward got 555 from this judge on I believe ribs might have been chicken and 9's and 8's from the other judges. THAT WAS the judge not the cook.

The low scoring judge is tossed out anyway, so why were they upset over the celebrity judge? Honest question, not being argumentative.

 

[Ron_L]

Mod Note:

Once again I have to step into a thread because folks can't stop the petty bickering!

Everyone is entitle to their opinion! If you disagree, threat them with some respect at least. If you can't do that, MOVE ON!

Side note: Please refrain from using red text. Red text is reserved fro the site admin and green it reserved for the mods.

 

[Okie Sawbones]

Mod Note:

Once again I have to step into a thread because folks can't stop the petty bickering!

Everyone is entitle to their opinion! If you disagree, threat them with some respect at least. If you can't do that, MOVE ON!

Side note: Please refrain from using red text. Red text is reserved fro the site admin and green it reserved for the mods.

You just used red text. :mrgreen: Sorry, I had no idea there were reserved colors on the forum. What am I going to do about OU now, come football season? :wink:

 

[DUBBAGA]

Side note: Please refrain from using red text. Red text is reserved fro the site admin and green it reserved for the mods.

My bad, just edited my use of red to blue

Side note: TDUFFY IS BACK :thumb: ... today is gonna be a great day

 

[bigdogphin]

You just used red text. :mrgreen: Sorry, I had no idea there were reserved colors on the forum. What am I going to do about OU now, come football season? :wink:

Looks like you are gonna have to FirePistols

 

[Ron_L]

You just used red text. :mrgreen: Sorry, I had no idea there were reserved colors on the forum. What am I going to do about OU now, come football season? :wink:

http://www.bbq-brethren.com/forum/showthread.php?t=14685

Post 2, near the bottom.

As far as OU, use an image?

:becky:

 

[bigdogphin]

You cannot average random events and produce meaningful data. Seating judges by average score is as effective as seating them by shoe size. Brad hit it right on the head. It's a standards of education problem coupled with a lack of enforced policies. Everyone on the same page should be the goal; not 8's across the board.

KCBS needs to get on top of this quickly and exert some authority as another more serious issue is morphing out of this problem. Some Reps and volunteers now seem to be taking it upon themselves to try and fix what they consider inequities in the judging tent by ignoring standard KCBS procedure and implementing their own personal "fix". This type of freelancing at contests is a huge concern. Even if a KCBS procedure is flawed it's a KNOWN process. Contests are not laboratories for people to experiment using the cook teams as lab rats because they feel something is unfair. All contests should be run the same - or as close as humanly possible. If a cook team adds red tipped lettuce or apple slices in their box because they think it "looks better" they get DQ's because it's a hard and fast rules violation (black letter law). Judges judge samples in the order they are placed on their judge's mat. (black letter law). Why are people being allowed to modify the turn in procedure? This needs to be addressed ASAP. KCBS needs to issue an edict that judging procedures will be followed to the letter and, if they aren't, there will be consequences.

Can I like this twice!!! Seating by scoring average will just drive scores up.. Reeducate the judges in the system and get everyone on the same page. Contests should be identical no matter where you compete or who the rep is. Get a group of cooks and judges together and come up with a judging and training system that is fair to everyone. This should be done at a committee level then to the board. The board keeps grasping at the highest flame without addressing the fire.

 

[Okie Sawbones]

http://www.bbq-brethren.com/forum/showthread.php?t=14685

And I suppose you think I read operating manuals as well. Just fire that chainsaw up and run with it. :mrgreen:

Post 2, near the bottom.

As far as OU, use an image?

:becky:

Thanks Ron. Appreciate it.

 

[Okie Sawbones]

Can I like this twice!!! Seating by scoring average will just drive scores up.. Reeducate the judges in the system and get everyone on the same page. Contests should be identical no matter where you compete or who the rep is. Get a group of cooks and judges together and come up with a judging and training system that is fair to everyone. This should be done at a committee level then to the board. The board keeps grasping at the highest flame without addressing the fire.

I agree with this, but would extend it to testing judges, not just raising your hand. Pass the test or try again. Then have to pass an online test every two years to remain certified. Probably a pipe dream, but I'd still like to see it.