PDA

View Full Version : Those dam judges


ModelMaker
10-26-2011, 08:50 AM
While keeping up with the BOD threads, discussions at contests, input from cook teams, one of the constant bones of contention is those damn judges and the lack of consistency between all of us.
I have given some thought to the problem (?) for some time now and have a couple suggestions to perhaps improve the perceived problems.

BOD candidates are touting continued education for the masses which I would look forward to, but how to implement it? One thought is since ALL CBJ's have to be active KCBS members and each have a renew date, why not incorporate a 10 - 15 question yearly test dealing with straightforward questions dealing with the KCBS rules and procedures (not like that goofy ass Masters test). Pass it you may judge next year, fail and not pass a second time, no card-no judge next year.

Secondly, since all CBJ's receive the Bullsheet why not use this platform to give a monthly topic of current issues to give all the CBJ's the same clarification at one time.

Third, have you ever read through the Rep advisory section at KCBS website? Anytime there is a major problem that requires a BOD decision it is added to the advisory vs a hard change in rule wording. Is there some reason a advisory section can't be set up for Judging problems?
Maybe it's being addressed now as it is and only needs to be presented to the masses more easily.
To me personally, a monthly Bullsheet column dealing with a current issue or rule, or action affecting judges and judging procedures that have a solution approved by the BOD would go a long way towards putting everybody on the same page and improve consistence, maybe.
Ed

tmcmaster
10-26-2011, 09:06 AM
Bravo!

Gadragonfly
10-26-2011, 09:21 AM
I would suggest that the changes come, not from a test or requirement for renewing your judging/membership, but rather from the reps. Judges are as good as their reps so perhaps we should retrain our reps. :shock:

Consider:

At every contest the reps have a printed list they are required to review with the judges and then they hit the play button on the CD player. At that point everyone’s eyes begin to glaze over and heads begin to nod or they begin checking their smart phones for current sports scores.

This was not the experience this year when my husband and I judged in Rancho Cordova, CA. The reps, Kathleen and Kelly McIntosh run a judges meeting that can only be described as a continuing education experience. Kelly; affectionately called Mrs. McIntosh by her judges (as in Yes, Mrs. McIntosh and No, Mrs. McIntosh); read one or two instructions and then opened the floor to the judges. She encouraged everyone to participate in the judges’ meeting, occasionally prompting the judges to provide information that promoted discussion and in turn the judges’ meeting became a training session for new judges and a refresher course for seasoned judges. We discussed the things that should not influence our scores and things that as individuals we like to see in a well presented box. We discussed written rules and debunked theories such as a thick slice of brisket meaning the meat was overcooked. I think we as a group discussed every point that is made on the CD. But most importantly, she and Kelly engaged their judges. I can’t describe the meeting other than to say it felt like a 3-dimentional experience rather than a flat lecture.

At almost every contest we have anywhere from 30 – 45 minutes between the judges’ meeting and the time we are expected to be back at our tables. Kathleen and Kelly McIntosh have found a way to use that extra time that benefits both the judges and ultimately the teams. I'd like to see this approach used in every judges meeting I attend.

Julie

ModelMaker
10-26-2011, 09:36 AM
I say put them in charge!!
One of the things I love about a Mike Lake run contest, is the control and order he demands. One of the things I hate about a Mike Lake run contest is his bias towards the cook team. There are two entities here and both need the same amount of attention.
Ed

YankeeBBQ
10-26-2011, 09:37 AM
I would suggest that the changes come, not from a test or requirement for renewing your judging/membership, but rather from the reps. Judges are as good as their reps so perhaps we should retrain our reps. :shock:

Consider:

At every contest the reps have a printed list they are required to review with the judges and then they hit the play button on the CD player. At that point everyone’s eyes begin to glaze over and heads begin to nod or they begin checking their smart phones for current sports scores.

This was not the experience this year when my husband and I judged in Rancho Cordova, CA. The reps, Kathleen and Kelly McIntosh run a judges meeting that can only be described as a continuing education experience. Kelly; affectionately called Mrs. McIntosh by her judges (as in Yes, Mrs. McIntosh and No, Mrs. McIntosh); read one or two instructions and then opened the floor to the judges. She encouraged everyone to participate in the judges’ meeting, occasionally prompting the judges to provide information that promoted discussion and in turn the judges’ meeting became a training session for new judges and a refresher course for seasoned judges. We discussed the things that should not influence our scores and things that as individuals we like to see in a well presented box. We discussed written rules and debunked theories such as a thick slice of brisket meaning the meat was overcooked. I think we as a group discussed every point that is made on the CD. But most importantly, she and Kelly engaged their judges. I can’t describe the meeting other than to say it felt like a 3-dimentional experience rather than a flat lecture.

At almost every contest we have anywhere from 30 – 45 minutes between the judges’ meeting and the time we are expected to be back at our tables. Kathleen and Kelly McIntosh have found a way to use that extra time that benefits both the judges and ultimately the teams. I'd like to see this approach used in every judges meeting I attend.

Julie

While that sounds like a wonderful experience, the CD's were actually produced to make all the judges meetings consistent so every judge at every contest received the same and correct information. In the past every meeting was different because they were presented by different reps. In my opinion there needs to be a happy medium. Yes the CD needs to be presented but the Reps also need to be given a little more freedom in the way the meetings are handled. If the reps are all properly trained it shouldn't be an issue. Of course that's a big if !

Gadragonfly
10-26-2011, 09:58 AM
While that sounds like a wonderful experience, the CD's were actually produced to make all the judges meetings consistent so every judge at every contest received the same and correct information. In the past every meeting was different because they were presented by different reps. In my opinion there needs to be a happy medium. Yes the CD needs to be presented but the Reps also need to be given a little more freedom in the way the meetings are handled. If the reps are all properly trained it shouldn't be an issue. Of course that's a big if !

They played the CD as well, they just augmented the CD with open discussion prior to playing the CD.

Julie

Leatherheadiowa
10-26-2011, 09:59 AM
Ed, from my view as a cook more often than a judge I think remedial education needs to be saved for those that exhibit poor judging qualities. One example I can think of is this: You are at a contest judging and at your table you judge chicken and the table captain glances at the scoresheets and says, "folks you were all real close on scores." or "we had a score that was off by quite a bit". If the table captain notices that that one judge is scoring away from the rest of the group's trend then maybe there could be some type of interaction taken so this judge that may be judging his/ her first contest in a few years or for whatever reason isn't following the judging criteria can be taken asside in a professional manner and re-educated. Maybe put a note in an account associated with their KCBS number and if there are more than a couple instances of this poor choice in judging habit then a disciplinary action could be taken.
I cooked in Omaha in September and we had a very nice score on our chicken until we got to judge 5, that judge had a two point variable in each judging criteria column. We had turned in with two other teams on the same six box tray that we knew and low and behold judge 5 had done the same to them in comparison to the other judges.
I am unsure of how to hold volunteer judges accountable without forcing them out and making them feel alienated. One way to help judges realize what they are implicating as a result of a high or low score is to force them to write out a comment. In Omaha I would have loved to have known what we could do different next time to improve our results, not take the comment as, "Why did they hate my chicken?!"
While I hate to add more oversight to what is supposed to be a hobby, I am sure Tippycanoe BBQ, Big Bob Gibson, and Smokin' Hoggz BBQ are thankful that in recent weeks they got a fair shake that matched the amazing quality of their BBQ.

YankeeBBQ
10-26-2011, 10:05 AM
They played the CD as well, they just augmented the CD with open discussion prior to playing the CD.

Julie

I was just looking at the Reps advisories and came across this.


Opinion: The CDs were produced so that the meeting instructions are repeated exactly the same at
every contest. Under no circumstances are KCBS Reps to deviate from the pre-recorded set of
instructions. If the CD fails to work, read the instructions, word-for-word. A copy of the
instructions should be available in your Contest Representative Manual (book), at KCBS.US, or on
the yahoo group.
You are not to offer your personal opinions! If a question about a rule is asked, you are to answer
the question to the best of your ability. Don’t offer opinions to judges on how to judge for
appearance, what tenderness means, etc.

Lake Dogs
10-26-2011, 10:11 AM
Mind you, I'm all for anything that will get more consistency in scoring. As a cook the variance will drive you bonkers! That said, part of the beauty is the subjective nature of it. Meaning that one guy likes hot, another guy likes it salty, another likes it sweet, and another likes the sauce burned on there is part of what makes this fun. However, when those scores come back the cook doesnt know what to make of them. It's frustrating (as a cook). I've always been a fan of the check-box feedback form. If it was too salty for judge 5 and 6, I'd like to know it. That tells me that judges 1-4 like it salty and 5-6 dont. etc. For that matter, I'd actually like feedback from every judge every time. That may be way too time consuming and too much extra effort from organizers, etc. However, 7 or below would be nice.

I mean, part of the beauty is in finding out what people like and what NOT to change as much as to TO change/fix.

Also this should remedy the compete nonsense comment cards that are seen WAY too often. You know the ones; the ones that presume they know how you've cooked something and they're suggesting that you do it different, rather than stating the facts. I've seen plenty where the meat was tough, but rather than just say that, the judge would say "needs to cook longer", or "needs to foil". I know of plenty of meat that was foiled and actually was cooked TOO long so it was tough. Telling me to "cook longer" doesn't tell me what the problem was. The check-box approach will.

Leatherheadiowa
10-26-2011, 10:19 AM
Mind you, I'm all for anything that will get more consistency in scoring. As a cook the variance will drive you bonkers! That said, part of the beauty is the subjective nature of it. Meaning that one guy likes hot, another guy likes it salty, another likes it sweet, and another likes the sauce burned on there is part of what makes this fun. However, when those scores come back the cook doesnt know what to make of them. It's frustrating (as a cook). I've always been a fan of the check-box feedback form. If it was too salty for judge 5 and 6, I'd like to know it. That tells me that judges 1-4 like it salty and 5-6 dont. etc. For that matter, I'd actually like feedback from every judge every time. That may be way too time consuming and too much extra effort from organizers, etc. However, 7 or below would be nice.

I mean, part of the beauty is in finding out what people like and what NOT to change as much as to TO change/fix.

Also this should remedy the compete nonsense comment cards that are seen WAY too often. You know the ones; the ones that presume they know how you've cooked something and they're suggesting that you do it different, rather than stating the facts. I've seen plenty where the meat was tough, but rather than just say that, the judge would say "needs to cook longer", or "needs to foil". I know of plenty of meat that was foiled and actually was cooked TOO long so it was tough. Telling me to "cook longer" doesn't tell me what the problem was. The check-box approach will.
AMEN BROTHER! I love the fact that you and I would both like to know the good things as well as the bad. I judged the contest in Mason City and I gave a comment card in each category to the entry that stood out. The quality of competition in Mason City was awesome and the entries I was priveledged to judged made it very tough and I wanted to make sure that the knew they impressed me and I appreciated their effort and knowledge.
The comment cards may be the best tool for improving and maintaining a cook's quality of entries. Comment cards take 30 seconds to fill out and for 30 seconds of a judges time they can immensely help competition.

Rookie'48
10-26-2011, 10:57 AM
I'm thinking that continuing education is better than remedial education because the first will eliminate the need for the second, for the most part.

How about this?: When something changes and/or needs clarification KCBS can publish it in the Bullsheet AND send out an e-mail blast to all members. If they can e-mail you an ad for T-shirts then there should be no problem sending "cook/judge advisories" just like the Reps get. In fact, why not send ALL such info out to ALL of the members at once? That way the Reps, cooks and judges are all on the same page at the same time.

Think about it: A little e-mail blast saying something like "It's come to our attention that a couple of judges are licking their fingers in between samples - please don't do that, it's unsanitary and disgusting" or "There have been more complaints about noise during quiet time - please respect your neighbor".

I think that this would be more effective than just issuing a Rep Advisory or putting it in the Bullsheet. What does it cost to send out something like that? Not much, I'll bet.

I also think that KCBS should encourage contest organizers to make the judging tent available on Friday evening for a "meet & greet" between judges & cooks. Invitations could be sent out to all registered cook teams and judges a week before the contest by e-mail. Some might want to turn this into a pot luck dinner, some might just go with a place to get together and meet each other. Will this work? I don't know, but I think that it's worth a try just to bring the two major groups at any comp closer.

I also like the idea of getting more judges involved on the cooking side of things. Maybe something like a pin for cooking with three different teams in a two year period? If more judges get to know the cooks more judges will stay for the awards to cheer on their new friends.

The annual or bi-annual test idea has it's merits, in fact it can even be done on-line at a very minimal cost. If more judges are knowledgeable about the rules, then we should have better, more consistent judging. If more judges have cooked with a team, then we have more judges who understand what goes into cooking a comp.

Leatherheadiowa
10-26-2011, 11:26 AM
Mind you, I'm all for anything that will get more consistency in scoring. As a cook the variance will drive you bonkers! That said, part of the beauty is the subjective nature of it. Meaning that one guy likes hot, another guy likes it salty, another likes it sweet, and another likes the sauce burned on there is part of what makes this fun. However, when those scores come back the cook doesnt know what to make of them. It's frustrating (as a cook). I've always been a fan of the check-box feedback form. If it was too salty for judge 5 and 6, I'd like to know it. That tells me that judges 1-4 like it salty and 5-6 dont. etc. For that matter, I'd actually like feedback from every judge every time. That may be way too time consuming and too much extra effort from organizers, etc. However, 7 or below would be nice.

I mean, part of the beauty is in finding out what people like and what NOT to change as much as to TO change/fix.

Also this should remedy the compete nonsense comment cards that are seen WAY too often. You know the ones; the ones that presume they know how you've cooked something and they're suggesting that you do it different, rather than stating the facts. I've seen plenty where the meat was tough, but rather than just say that, the judge would say "needs to cook longer", or "needs to foil". I know of plenty of meat that was foiled and actually was cooked TOO long so it was tough. Telling me to "cook longer" doesn't tell me what the problem was. The check-box approach will.

I'm thinking that continuing education is better than remedial education because the first will eliminate the need for the second, for the most part.

How about this?: When something changes and/or needs clarification KCBS can publish it in the Bullsheet AND send out an e-mail blast to all members. If they can e-mail you an ad for T-shirts then there should be no problem sending "cook/judge advisories" just like the Reps get. In fact, why not send ALL such info out to ALL of the members at once? That way the Reps, cooks and judges are all on the same page at the same time.

Think about it: A little e-mail blast saying something like "It's come to our attention that a couple of judges are licking their fingers in between samples - please don't do that, it's unsanitary and disgusting" or "There have been more complaints about noise during quiet time - please respect your neighbor".

I think that this would be more effective than just issuing a Rep Advisory or putting it in the Bullsheet. What does it cost to send out something like that? Not much, I'll bet.

I also think that KCBS should encourage contest organizers to make the judging tent available on Friday evening for a "meet & greet" between judges & cooks. Invitations could be sent out to all registered cook teams and judges a week before the contest by e-mail. Some might want to turn this into a pot luck dinner, some might just go with a place to get together and meet each other. Will this work? I don't know, but I think that it's worth a try just to bring the two major groups at any comp closer.

I also like the idea of getting more judges involved on the cooking side of things. Maybe something like a pin for cooking with three different teams in a two year period? If more judges get to know the cooks more judges will stay for the awards to cheer on their new friends.

The annual or bi-annual test idea has it's merits, in fact it can even be done on-line at a very minimal cost. If more judges are knowledgeable about the rules, then we should have better, more consistent judging. If more judges have cooked with a team, then we have more judges who understand what goes into cooking a comp.
I like your ideas in regards to having judges cook with a team. That may be a good way to get them to see things from both sides of the turn in table. I wish I had more faith that putting a statement out about things noticed in judging that could be improved would work but I feel that it won't. Working in medicine I can tell you that doctors, nurses, and other licensed professionals get notices frequently to help them and protect the well being of patients. We get Best Practices and Lessons Learned bulletins that often describe where something went horribly wrong, what happened, how it happened, why it happened, and how it could have been prevented. 99% of the time those bulletins are tossed aside, trashed, or filed without any review, and this is human life and well being, not BBQ.
Humans by nature don't like to be criticized and our busy lifestyles don't make reading the Bull Sheet a priority or routine email messages. Online education is a great tool and I use it with students of mine when I teach, the only thing is that someone always prints off the test, quiz, or handbook and shares it. Then that education is useless and we are back to square one.
While there is no clean cut solution one thing that seems to work is having some type of negative response given when bad behavior is exhibited by humans. I will leave the stonings and lashings to the folks on the KCBS rules committee.

Bigmista
10-26-2011, 11:57 AM
I would suggest that the changes come, not from a test or requirement for renewing your judging/membership, but rather from the reps. Judges are as good as their reps so perhaps we should retrain our reps. :shock:

Consider:

At every contest the reps have a printed list they are required to review with the judges and then they hit the play button on the CD player. At that point everyone’s eyes begin to glaze over and heads begin to nod or they begin checking their smart phones for current sports scores.

This was not the experience this year when my husband and I judged in Rancho Cordova, CA. The reps, Kathleen and Kelly McIntosh run a judges meeting that can only be described as a continuing education experience. Kelly; affectionately called Mrs. McIntosh by her judges (as in Yes, Mrs. McIntosh and No, Mrs. McIntosh); read one or two instructions and then opened the floor to the judges. She encouraged everyone to participate in the judges’ meeting, occasionally prompting the judges to provide information that promoted discussion and in turn the judges’ meeting became a training session for new judges and a refresher course for seasoned judges. We discussed the things that should not influence our scores and things that as individuals we like to see in a well presented box. We discussed written rules and debunked theories such as a thick slice of brisket meaning the meat was overcooked. I think we as a group discussed every point that is made on the CD. But most importantly, she and Kelly engaged their judges. I can’t describe the meeting other than to say it felt like a 3-dimentional experience rather than a flat lecture.

At almost every contest we have anywhere from 30 – 45 minutes between the judges’ meeting and the time we are expected to be back at our tables. Kathleen and Kelly McIntosh have found a way to use that extra time that benefits both the judges and ultimately the teams. I'd like to see this approach used in every judges meeting I attend.

Julie

Kelly and Kathleen (Kathleen is Mrs. McIntosh, not Kelly but I will tell him what you said) are the most respected reps on the west coast. They are honest, likeable, highly engaging and have integrity beyond reproach. They take time to get to know the teams as well as the judges. They also invented the Southwest shuffle system for seating judges.

They are two of my favorite people and both judges and reps could learn a lot from them.

BTW...they hand out a list of phone numbers to all of the teams at a contest.

1. Security
2. Contest Organizer
3. Kelly & Kathleen

They have a rule that if any situation gets so bad that they get a call, somebody is going home. Period.

Stoke&Smoke
10-26-2011, 12:12 PM
I say put them in charge!!
One of the things I love about a Mike Lake run contest, is the control and order he demands. One of the things I hate about a Mike Lake run contest is his bias towards the cook team. There are two entities here and both need the same amount of attention.
Ed

Just curious what you mean by Mike's "bias towards the cook team".

I've judged a number of contests he's repped, but cooked more, are you saying he favors teams over judges? Not trying to start anything here, just want to understand what you're saying.

Not to slam judges, as without them, there can be no comp really. (and I am one, as well as a certified table captain) But shouldn't the cooks be given the highest level of consideration?

The judges are there to judge the product the cooks made. The cooks are a lot more invested financially, timewise, etc. The judges pay their transportation and possibly lodging costs.

So do the cooks, along with entry fees, meat costs, ingredient costs, etc. So doesn't it follow that it's the cooks, whose product is being judged, deserve the utmost consideration?

I'm always happy to see Mike and Theresa repping. I know we have the best chance at a fair shake

Jorge
10-26-2011, 12:17 PM
While keeping up with the BOD threads, discussions at contests, input from cook teams, one of the constant bones of contention is those damn judges and the lack of consistency between all of us.
I have given some thought to the problem (?) for some time now and have a couple suggestions to perhaps improve the perceived problems.

BOD candidates are touting continued education for the masses which I would look forward to, but how to implement it? One thought is since ALL CBJ's have to be active KCBS members and each have a renew date, why not incorporate a 10 - 15 question yearly test dealing with straightforward questions dealing with the KCBS rules and procedures (not like that goofy ass Masters test). Pass it you may judge next year, fail and not pass a second time, no card-no judge next year.

Secondly, since all CBJ's receive the Bullsheet why not use this platform to give a monthly topic of current issues to give all the CBJ's the same clarification at one time.

Third, have you ever read through the Rep advisory section at KCBS website? Anytime there is a major problem that requires a BOD decision it is added to the advisory vs a hard change in rule wording. Is there some reason a advisory section can't be set up for Judging problems?
Maybe it's being addressed now as it is and only needs to be presented to the masses more easily.
To me personally, a monthly Bullsheet column dealing with a current issue or rule, or action affecting judges and judging procedures that have a solution approved by the BOD would go a long way towards putting everybody on the same page and improve consistence, maybe.
Ed

I think you offer a good place to start.

My understanding is that there is a 'continuing education' program that is in the process of being put together that would be available online for members, not just CBJs. As long as that is kept current, I think it could be a great resource for ALL.

Some comments have touched on Rep Advisories. There is no reason for them not to be available, online and in one package for all members. If I'm a member and I miss a board meeting where a particular issue was discussed and some advisory issued, I'm out of luck. Why can't we have a system where that information is sent out in a blast, BUT give the member the option to opt out if they don't want to receive future communication of that sort. If it's up to the cook/judge/rep/or organizer to opt out...their ignorance of the rule/advisory is THEIR responsibility.

As far as everyone being in this together, I agree! I think it's important to realize that 99% of the complaints about judges are probably directed at 1% of them. Some issues are based on perception and not reality, some are based on cooks not recognizing they had a bad day, and some are because some judges just didn't follow the rules. KCBS is in the process of developing and deploying new scoring software that will track judges. Given time I think this will be a great tool. I say given time because based on experience with my day job I spend a lot of time crunching numbers, to include consulting with professors at institutions like MIT and Stanford. It takes time for numbers to tell a story. Once that story becomes clear it's possible to address individual issues with education or corrective action. Given time I think that will even out some issues, and reduce some of the judge bashing...but it won't ever go away. Just as some judges will always feel the need to offer advice, sometimes at exactly the wrong time:wink:

Right now, there are several generations of judges out there and their instruction was a little different with each generation. In addition an example given in this thread makes it clear that a give and take session at a judges meeting could potentially skew the current standard further based on interpretation. Standardization benefits everyone by reinforcing criteria that's the same from contest to contest. Everybody gets a fair shake that way.

Whether I'm elected or not, I'd like to see you serve on the rules committee Ed. I don't always agree with you, but you are a great advocate for judges.

Gadragonfly
10-26-2011, 12:18 PM
[QUOTE=Bigmista;1830210]Kelly and Kathleen (Kathleen is Mrs. McIntosh, not Kelly but I will tell him what you said)

That's what I get for cutting and pasting from something I wrote this summer (and then never sent to the Bullsheet). In its original form, I had the right name for "Mrs. McIntosh" :crazy:

J

ModelMaker
10-26-2011, 05:34 PM
Just curious what you mean by Mike's "bias towards the cook team".

I've judged a number of contests he's repped, but cooked more, are you saying he favors teams over judges? Not trying to start anything here, just want to understand what you're saying.

Not to slam judges, as without them, there can be no comp really. (and I am one, as well as a certified table captain) But shouldn't the cooks be given the highest level of consideration?

The judges are there to judge the product the cooks made. The cooks are a lot more invested financially, timewise, etc. The judges pay their transportation and possibly lodging costs.

So do the cooks, along with entry fees, meat costs, ingredient costs, etc. So doesn't it follow that it's the cooks, whose product is being judged, deserve the utmost consideration?

I'm always happy to see Mike and Theresa repping. I know we have the best chance at a fair shake

The "bias" I speak of is the exact thing you are presenting. Cooks do this, cooks spend this, cooks deserve this.
Cooks decide to practice their skills at a BBQ event (period).
Judges decide to come to this event and practice their skill also (period).
My preference is to judge a Mike and Theresa contest because of how they run it. It's obvious that Mike the cook has a preference, it's not good, it's not bad, it is just there.
Ed

Slamdunkpro
10-26-2011, 05:52 PM
It's a deeper problem than a few rogue judges or a continuing education program. As BBQ gets into bigger and bigger money there are people that will take advantage of the ambiguities in the rules. The rules committee needs to take a deep look at the rules and eliminate the loopholes, ambiguities, and open windows. Once that's done a lot of judging issues will resolve themselves.

There is a lot of talk about monitoring judges for "outside the norm" scoring. This is a slippery slope that can easily lead to score inflation if not managed in the proper manner. Go down the wrong road and you wind up with 50 180's at every contest.

Comment cards need to be used at every event and not censored by the reps. It would be easy to make comment cards more productive without sacrificing speed by using a sliding scale mark system. The judge simply marks where they feel where the sample is lacking and to what degree

As an example: (judge marks in red)

TOO SALTY............................GOOD............. .........NEEDS SALT
|-----------------|--------------|------------------------------|
MUSHY................................GOOD......... .....................TOUGH
|--------------------------------|-------------------------|----|
SPICY.................................. GOOD............................. BLAND
|--|-----------------------------|------------------------------|
TOO SWEET...........................GOOD.............. ........TOO ACIDIC
|--------------------------------|-------------------------|----|

Based on this comment card this cook's chicken was undercooked, salty, too hot and vinegary.

Bentley
10-26-2011, 06:58 PM
How about this?: When something changes and/or needs clarification KCBS can publish it in the Bullsheet AND send out an e-mail blast to all members.

I think that this would be more effective than just issuing a Rep Advisory or putting it in the Bullsheet.


I have been a KCBS member & CBJ for 10 year. I have read the Bullsheet maybe 5 times in that period...I check my email 10 times a day.

Good Idea.

Bunny
10-26-2011, 08:38 PM
I was just looking at the Reps advisories and came across this.


Opinion: The CDs were produced so that the meeting instructions are repeated exactly the same at
every contest. Under no circumstances are KCBS Reps to deviate from the pre-recorded set of
instructions. If the CD fails to work, read the instructions, word-for-word. A copy of the
instructions should be available in your Contest Representative Manual (book), at KCBS.US, or on
the yahoo group.
You are not to offer your personal opinions! If a question about a rule is asked, you are to answer
the question to the best of your ability. Don’t offer opinions to judges on how to judge for
appearance, what tenderness means, etc.

Thank you, Steve. Exactly what we've been told to do. Sometimes we reiterate the procedures, but that's important that all new judges understand what's happening. (Like getting the numbers on the right square). But never give our opinion on what's a good piece of brisket. If that was the case, I'd tell them how I cook mine. LOL:becky:

TooSaucedToPork
10-27-2011, 09:51 AM
The "bias" I speak of is the exact thing you are presenting. Cooks do this, cooks spend this, cooks deserve this.
Cooks decide to practice their skills at a BBQ event (period).
Judges decide to come to this event and practice their skill also (period).
My preference is to judge a Mike and Theresa contest because of how they run it. It's obvious that Mike the cook has a preference, it's not good, it's not bad, it is just there.
Ed

I take offense to this.

There should be a bias to the cooks (period).

Any idiot with $75 dollars can pay for a judging class and judge my food which I have spent thousands on equipment, food, entry fees and lodging. A judge has invested $75 in a class and possibly a hotel for the contest, I have spent more than a judge just entering a contest.

The judges need to be educated better to understand the "free" food they score (and carry away with them in their cute little coolers) is actually my blood, sweat, and tears. Yes there is a bias to the cooks, anyone can pay $75 and become a judge...how many people can cook competition quality BBQ...not many.

I'm not trashing judges (I am one) they are needed to make this sport work. I'm just saying that, yes judges are important, but the cooks (and good reps) are the stars of the show...People don't pay a gate fee to see a bunch of judges.

I'm tired of spending my hard earned money on a contest only to be thrown out of the top 5 by that dang judge #5...(its always that guy). We as cooks are doing our part...many judges need to be better trained to do theirs.

Sorry for the rant, its a topic that has needed fixing for years...IMHO

Bentley
10-27-2011, 11:28 AM
I take offense to this.

Any idiot with $75 dollars can pay for a judging class and judge my food which I have spent thousands on equipment, food, entry fees and lodging. A judge has invested $75 in a class and possibly a hotel for the contest, I have spent more than a judge just entering a contest.



Maybe the teams that have spent the most on equipment and supplies need to be given more points for that effort. Maybe if I know who has spent the most on equipment and supplies I can judge more accordingly...just sayin.

PorkQPine
10-27-2011, 11:29 AM
"Any idiot with $75 dollars can pay for a judging class and judge my food which I have spent thousands on equipment, food, entry fees and lodging. A judge has invested $75 in a class and possibly a hotel for the contest, I have spent more than a judge just entering a contest.

The judges need to be educated better to understand the "free" food they score (and carry away with them in their cute little coolers) is actually my blood, sweat, and tears. Yes there is a bias to the cooks, anyone can pay $75 and become a judge...how many people can cook competition quality BBQ...not many.

I'm not trashing judges (I am one) they are needed to make this sport work."

I guess you define trashing different than I do. BTW, it takes more than one judge to toss you out of the top 5. After 5 years judging and cooking with a team once a year, i can tell you that I have seen many cases where more than one sample in the box was not good. This is especially true with ribs and chicken since they are separate pieces. I also see burnt ends ruin a turn-in occasionally when the burnt ends are not as good as the slices and the same goes for pork where the pulled and chunks are not of the same quality. I know almost every CBJ in N.Cal and the reps who cover our territory and I can tell you that they are serious and do their best.

I have been an advocate of continuing education for CBJ's and also CBJ's should cook with a team at least once a year. Once the teams get to know the judges I think that a lot of the animosity directed toward the judges will diminish.

Rookie'48
10-27-2011, 11:34 AM
...Any idiot with $75 dollars can pay for a judging class and judge my food which I have spent thousands on equipment, food, entry fees and lodging.

...the "free" food they score (and carry away with them in their cute little coolers) is actually my blood, sweat, and tears. Yes there is a bias to the cooks, anyone can pay $75 and become a judge...how many people can cook competition quality BBQ...not many.

I'm not trashing judges (I am one)

Ok, I guess that I'm one of the "idiots" that paid for a judging class in early 2007. I was a part-time member of a cooking team & I wanted to judge maybe two or three comps a year while cooking five or six. An injury to my lower back put my competitive cooking out of the question, at least for any more than one or two comps a year, so I decided to spend more of my time judging.

I have judged over sixty contests in the past five years, the vast majority of them 100 miles or more from home. I normally take my travel trailer to an RV park near the comp and show up on the grounds around 6 pm to visit with old friends and meet new ones. Yes, I have a "cute little" cooler that I pull around on a two-wheeler. Mine usually has Busch Light and a whole bunch of different smoked cheeses in it that I'm quite willing to share as I wander around. By the way, my cooler has yet to see the inside of a judging tent.

I usually leave the contest grounds around 11:30 pm after leaving my two-wheeler / cooler stashed at a friend's spot. In the morning I'm back to judge some damn fine (for the most part) BBQ. As I walk into the grounds I wave and say "Good morning & good luck" to any cook who looks my way, whether I know them or not.

After the judging is over I'm back out on the grounds visiting with friends, answering questions about how the food was, maybe sharing a beer or giving my opinion on their leftovers. When it's time for the awards I'm right there cheering on all of the competitors, but especially those that I know personally.

And as far as cooking ability goes, I know that I can do better than some of the turn-ins that I've seen. The majority of comp cooks can cook circles around me and that's a fact - but there are a few ...

So yeah, I guess that you've got more dollars wrapped up in this hobby than I do. But I'll be damned if you've got more time or commitment invested than I do.

Warthog
10-27-2011, 11:41 AM
Maybe the teams that have spent the most on equipment and supplies need to be given more points for that effort. Maybe if I know who has spent the most on equipment and supplies I can judge more accordingly...just sayin.
Now that is a load of crap. Judge on how much a team spent. Food should be judged on its merits only. Your saying the guy with the Jambo should get higher points then the the guy using a WSM?

Jeff_in_KC
10-27-2011, 11:49 AM
Now that is a load of crap. Judge on how much a team spent. Food should be judged on its merits only. Your saying the guy with the Jambo should get higher points then the the guy using a WSM?

I think you missed the sarcasm tags. :wink:

This is an interesting thread. I think if anyone uses the term "remedial", we should turn and run...FAST! What better way to alienate judges than tell them they are now having to take "Remedial Judging"? What an insult! I am all for continuing education for CBJs. We have a problem with generations of judges. This was a point I made while campaigning in 2009. We have long time judges who learned years ago under different circumstances and sets of rules by totally different people than are teaching now. All judges need to be on the same page and there are ways we can do that. It's not remedial at all. I will say this though - getting upset that one judge was two points off everyone else is no cause for alarm. If I'm getting 8's and 9's and one gives me a 7 or two, that's the breaks. Sorry but it's the truth. Now if I'm getting 6's or below from one judge when everything else is am 8 or 9, that's a little different. Good food is good food and most judges will recognize that. I think the issue is that some judges just think that because of the definitions assigned to the scoring numbers, giving you a seven is paying you a compliment. It's above the average but not outstanding. And we don't all cook outstanding food every week. Maybe we like to THINK we do but if you're honest with yourself, you'll have to admit there were probably things you could have done better to get a better score. I have blamed judges before but in the last two years, we've really learned a lot about what we're turning in. And 95% of the time, judges get it right. Just sayin'...

Rookie'48
10-27-2011, 11:56 AM
Maybe the teams that have spent the most on equipment and supplies need to be given more points for that effort. Maybe if I know who has spent the most on equipment and supplies I can judge more accordingly...just sayin.

Now that is a load of crap. Judge on how much a team spent. Food should be judged on its merits only. Your saying the guy with the Jambo should get higher points then the the guy using a WSM?

I think you missed the sarcasm tags. :wink:

Bentley ..... sarcastic? ..... say it ain't so! :thumb:

Bentley
10-27-2011, 11:57 AM
Now that is a load of crap. Judge on how much a team spent. Food should be judged on its merits only. Your saying the guy with the Jambo should get higher points then the the guy using a WSM?

I think you missed the sarcasm tags. :wink:

Ding,ding,ding...We have a winner!

Slamdunkpro
10-27-2011, 12:00 PM
It really needs to stop being an us vs them thing. Sure, cooks spend more money, but they also have the oppurtunity for a greater financial return. The only real compensation that a judge gets is possibly a goodie bag and left overs (unless it's a comp that prohibits that - but that's another pot to stir).

Jorge
10-27-2011, 12:33 PM
I think you missed the sarcasm tags. :wink:

This is an interesting thread. I think if anyone uses the term "remedial", we should turn and run...FAST! What better way to alienate judges than tell them they are now having to take "Remedial Judging"? What an insult! I am all for continuing education for CBJs. We have a problem with generations of judges. This was a point I made while campaigning in 2009. We have long time judges who learned years ago under different circumstances and sets of rules by totally different people than are teaching now. All judges need to be on the same page and there are ways we can do that. It's not remedial at all. I will say this though - getting upset that one judge was two points off everyone else is no cause for alarm. If I'm getting 8's and 9's and one gives me a 7 or two, that's the breaks. Sorry but it's the truth. Now if I'm getting 6's or below from one judge when everything else is am 8 or 9, that's a little different. Good food is good food and most judges will recognize that. I think the issue is that some judges just think that because of the definitions assigned to the scoring numbers, giving you a seven is paying you a compliment. It's above the average but not outstanding. And we don't all cook outstanding food every week. Maybe we like to THINK we do but if you're honest with yourself, you'll have to admit there were probably things you could have done better to get a better score. I have blamed judges before but in the last two years, we've really learned a lot about what we're turning in. And 95% of the time, judges get it right. Just sayin'...

Now that is a load of crap. Judge on how much a team spent. Food should be judged on its merits only. Your saying the guy with the Jambo should get higher points then the the guy using a WSM?

Maybe the teams that have spent the most on equipment and supplies need to be given more points for that effort. Maybe if I know who has spent the most on equipment and supplies I can judge more accordingly...just sayin.

It really needs to stop being an us vs them thing. Sure, cooks spend more money, but they also have the oppurtunity for a greater financial return. The only real compensation that a judge gets is possibly a goodie bag and left overs (unless it's a comp that prohibits that - but that's another pot to stir).

Bentley is known for going over the top, from time to time, to make a point. I don't always agree with the method or the point, but he's put some thought into it and is willing to stand behind it.

Jeff made a point that I strongly agree with. Judges usually get it right. Judge 5 isn't always wrong. Sometimes they get a rib that looked as good as the others, but just wasn't! It happens. It's not like a pork box where you can pull a little piece off of everything going into the box to make sure it has the right flavor etc... With the new software, given time, it will be possible to identify those judges that are consistently scoring outside of the norm and then address those issues. The overwhelming majority of judges take the task seriously, and do a phenomenal job. In Texas we don't use CBJs, but draw people off the street. They usually get it right as well. What KCBS offers is a little more consistency, and that's something I think we need to focus on and improve where possible.

Slamdunkpro, raises a fair point as well. Judges don't have the financial investment a cook does, but they've invested their time and deserve courtesy and respect for their contributions as well. If you think there are regional flavor profiles in KCBS then I invite you to come cook in Texas or talk to Texas cooks about what changes some make when cooking at a VFW etc...

99% of the issues that cooks have, are probably with 1% of the judges. Cooks and judges alike should probably think about that before speaking, and I'll raise my hand and admit that I've been guilty in the past. You can't judge without cooks, but a cook is going to be be happier more often than not with an experienced judge.

Bentley
10-27-2011, 01:21 PM
And we don't all cook outstanding food every week. Maybe we like to THINK we do but if you're honest with yourself, you'll have to admit there were probably things you could have done better to get a better score.

Bitter pill to swollow, but so true! Specially when a a team member tells you!

Bigdog
10-27-2011, 02:12 PM
One thought is since ALL CBJ's have to be active KCBS members and each have a renew date, why not incorporate a 10 - 15 question yearly test dealing with straightforward questions dealing with the KCBS rules and procedures (not like that goofy ass Masters test). Pass it you may judge next year, fail and not pass a second time, no card-no judge next year.


You do bring up some good points Ed, but this will NEVER happen as KCBS's "dirty little secret" is that the judges membership dues is the CASH COW of the KCBS. Now while I'm reletively sure that they would LIKE to have something like this happen, it never will IMHO.

TooSaucedToPork
10-27-2011, 02:17 PM
Maybe the teams that have spent the most on equipment and supplies need to be given more points for that effort. Maybe if I know who has spent the most on equipment and supplies I can judge more accordingly...just sayin.

I didn't mean it the way it sounds.

We have alot invested...more than judges. Even the guy with a homebuilt UDS has more invested...

I'm talking about the judges that need a continuing education course. Or the few that have a high and mighty attitude (we've all met them) that think they know it all. No one knows it all, none of us ever will as this sport is a constantly changing monster.

That is why education is important...

Rookie - you are probably a shining example of a great judge. You hang with the teams, get to know us, follow the rules, and judge based on experience, education, and rules...

From what I've seen around here, Half (maybe a little less) of judges don't do this.

It was a generalization and I am sorry. In my area of the country (West New York) We have an overabudance of judges, many very new, in need of education. I have taken off my team shirt and gone and stood next to the judging tent as they were exiting judging for the day, listening to conversations.

At the last contest I did this - There were 4 people (all CBJ's) saying how none of their entries tasted like Famous Dave, or Smokey Bones, or (insert restaurant here), so I marked them as a six...or a seven. I heard two more judges, wives that took a class to come along w their husbands.

Wife 1 - "I don't really LIKE PORK...I just give it all 7's, sometimes slip an 8 in there, or I look at Daniel's sheet and write what he does."
Wife 2 - "Thats how I feel about the steak they gave us, I do the same with Charles"

I thought..Steak??? its brisket...WTF?!?! She doesn't even know what she is judging

I STORMED AWAY...Thats 6 people I overheard that should not be judging. At least not judging without MORE EDUCATION. Thats 5 CBJ's (all but 1 had nametags) that screwed alot of teams.

We as cooks constantly practice and educate ourselves...The judges should be given the same opportunity. I say opportunity because it benifits everyone in the organization.

I'm not trashing judges, just saying there needs to be a high standard and a way to ensure that high standard is maintained

Stoke&Smoke
10-27-2011, 02:31 PM
The "bias" I speak of is the exact thing you are presenting. Cooks do this, cooks spend this, cooks deserve this.
Cooks decide to practice their skills at a BBQ event (period).
Judges decide to come to this event and practice their skill also (period).
My preference is to judge a Mike and Theresa contest because of how they run it. It's obvious that Mike the cook has a preference, it's not good, it's not bad, it is just there.
Ed

Ed, I have to respectfully disagree. It is a BBQ cooking competition. As with any competion that involves judges, (olympics, pillsbury bake off, and everything in between) the contest is about those competing, not the judges, however important the judges may be to the event.

So the governing body, and their representatives owe it to the competitors to ensure they are judged as fairly as possible. There is a slant, and there should be.

I believe what many others have said already, there is a very small minority of judges causing issues, confusion, etc. I think that tracking would enable KCBS to determine which judges are consistently out of synch with their fellow judges, if the software used has that funtionality.

I haven't judged in 4 years, and would certainly want to attend some sort of refresher were I to consider it.

TooSaucedToPork
10-27-2011, 03:02 PM
The fact of the matter is, Cooks pay to be at a contest, and expect good judges. The best and most well trained judges possible.

If a cook turns in bad food it hurts that cook.

If there are bad judges, or poorly trained judges it hurts the 4,5,or 6 Teams entries they judged. And that uneducated judge just judged at least 4 catagories.Thats 16, 20, or 26 teams that just got screwed out of a low dropped score. Yeah the arguement is there that that low dropped score takes care of bad judges, that is wrong. It is supposed to be there to get rid of a low opinion of that entry by a good well trained judge that just didn't like it.

I like the Olympics reference earlier. WATCH THE SCORES. They are educated judges, and the scores are all very close, they just drop the lowest score. It is not 2-3 full points under the other scores.

Educated and Fair judges are important...

Jorge
10-27-2011, 03:26 PM
I like the Olympics reference earlier. WATCH THE SCORES. They are educated judges, and the scores are all very close, they just drop the lowest score. It is not 2-3 full points under the other scores.

Educated and Fair judges are important...

In fairness, while that Olympic judging is also subjective, it's a single performance being judged. In BBQ each judge gets a separate sample. It's unrealistic to expect each rib to be the same, each and every time.

I understand your point, but the analogy doesn't carry over to BBQ.

Bentley
10-27-2011, 03:35 PM
At the last contest I did this - There were 4 people (all CBJ's) saying how none of their entries tasted like Famous Dave, or Smokey Bones, or (insert restaurant here), so I marked them as a six...or a seven. I heard two more judges, wives that took a class to come along w their husbands.


Those are down right scary statements, BBQ Restaurants must be a lot better in Western New York then they are out here in SoCal!


Wife 1 - "I don't really LIKE PORK...I just give it all 7's, sometimes slip an 8 in there, or I look at Daniel's sheet and write what he does."
Wife 2 - "Thats how I feel about the steak they gave us, I do the same with Charles"


Not sure if Daniel & Charles are spouse or significant others, and I am assuming Wife 1 & 2 are buddies, if those statements are true, under no circumstances should the spouces be sitting together, and we are advised by the Reps here in CA that they would prefer buddies not sit together either.

So it sounds like your Reps are dropping the ball on those examples!

TooSaucedToPork
10-27-2011, 03:42 PM
In fairness, while that Olympic judging is also subjective, it's a single performance being judged. In BBQ each judge gets a separate sample. It's unrealistic to expect each rib to be the same, each and every time.

I understand your point, but the analogy doesn't carry over to BBQ.

I understand your point, and acknowledge the truth in it.

I'm talking about tracking judge scoring like they do in the olympics. If a judge is not fair or is not judging by the rules, they no longer judge...or are re-educated. The point I was trying to make was that well-educated judges scores are usually consistent. Yes, sometimes there is a wiley piece of chicken that is terrible. But if you see that a judge is consistantly scoring different than the others there needs to be a conversation. You gotta find the outliers and re-educate or eliminate them to give all teams a fair shot.

I know this is probably impossible, but it would be a great day for KCBS...

TooSaucedToPork
10-27-2011, 03:54 PM
Not sure if Daniel & Charles are spouse or significant others, and I am assuming Wife 1 & 2 are buddies, if those statements are true, under no circumstances should the spouces be sitting together, and we are advised by the Reps here in CA that they would prefer buddies not sit together either.

So it sounds like your Reps are dropping the ball on those examples![/QUOTE]

This conversation I overheard just outside the beer tent. The Ladies were waiting on glasses on wine brought over by their husband/significant other. Now whether they were talking about the contest I was at or another contest I do not know, but that was the conversation. The reps we had were awesome, and I don't believe they would have allowed buddies/S.O's to sit together...but things can slip by you when you deal with 60-70 judges

Slamdunkpro
10-27-2011, 04:25 PM
Ya know,,,,,,,,,

I've been reading this and the other BoD / KCBS fix threads for a while now and after the last couple of posts something jumped out at me.

A lot of the issues that cooks/judges/organizers are having seem to be Rep issues. Examples:

In the above posts - buddies & SO's seated at the same table
Pork collars & the 5lb rule - when was the last time you saw a rep with a scale?
Judges not liking pork / chicken skin / whatever = low scores - where is the rep action at the event?
Inconsistent judging instruction
Parting pork and putting in back in the cooker - When have you ever seen a rep spot check a cooker?
Incorrect application of the rules

Don't get me wrong, I know a number of reps and most are hard working, knowledgeable & involved in the event, but I have seen some that just go through the motions and never leave the scoring / judging area. Maybe the new BOD should add rep performance tracking & rep remedial training tests to their agenda. They could look at how many issues contests have and who the rep is - maye there's a pattern.


Serious post.

Rookie'48
10-27-2011, 05:05 PM
In the above posts - buddies & SO's seated at the same table

Pork collars & the 5lb rule - when was the last time you saw a rep with a scale?
Judges not liking pork / chicken skin / whatever = low scores - where is the rep action at the event?
Inconsistent judging instruction
Parting pork and putting in back in the cooker - When have you ever seen a rep spot check a cooker?
Incorrect application of the rules
Maybe the new BOD should add rep performance tracking & rep remedial training tests to their agenda. They could look at how many issues contests have and who the rep is - maybe there's a pattern.
It seems to me that one way to correct most, if not all of these problems is by education. Letting ALL members know what the rules are & when they're changed would go a long way in solving most issues. Not sitting with your spouse, SO, etc is well known but sometimes I see mother/daughter, brother/sister, etc. sitting at the same table. What about "buddies"? I've met a lot of judges in the past five years, am I allowed to sit with any of them or do I try to find a table where I don't know any one?

Maybe an occasional e-mail blast to ALL members (cooks, judges, Reps, organizers, every one) to remind us of the rules - like you're going to eat pork & chicken skin, it's a BBQ contest!

As far as the five pound rule, the meat should have been checked at the meat inspection. As far as the "parting" rule, I think that it should be eliminated but don't get me going on that one :doh:.

Consistent judging is only going to come if there is consistent teaching and some sort of continuing education.

Incorrect (?) application of the rules and tracking Reps is another problem, but we have had Reps disciplined by the BoD with some Reps getting suspensions or even being removed as a KCBS Contest Rep in extreme cases. I think that this process can be improved upon, but the basis is there.

ModelMaker
10-27-2011, 05:09 PM
You know, all of you (us,I'm one also) cook teams want to cook a event and have your efforts judged fairly by well trained judges.
I as a judge I only want a way to weed out the judges that don't share the same thought process and guidlines. We all see them in the tent. If KCBS can't figure out a way to identify and remedy the problem there is always going to be this divide.
I don't like it.
Ed

Fat Freddy
10-27-2011, 05:33 PM
One thing to consider is some KCBS contests(if not many) do not have 100% certified judges, so occasionally you may get a table with a judge that doesnt know as much as we all would hope they do. At one time I fell into this category but felt that I was enjoying judging so I should become a "trained" judge. I took the class and have been proud to be a certified judge. This year I felt the next logical step was to compete. I have yet to be very successful but I have not embarrassed myself either but I have learned alot that will make me a better judge when I cannot afford to compete.

Alexa RnQ
10-27-2011, 05:40 PM
Pork collars & the 5lb rule - when was the last time you saw a rep with a scale?
Actually, during this last season! Kelly and Kathleen MacIntosh had one. And, by the way, they do their own meat inspection -- not like some contests where it's delegated to someone who simply glances in a cooler for the presence of ice, as happened at a different contest we attended.

I only want a way to weed out the judges that don't share the same thought process and guidlines. We all see them in the tent.

This is perfectly answered by another circular argument, that levied against continuing education and mandatory comment cards below a certain score -- "It'll discourage participation by judges!" If marking a card or reading advisories is too arduous a task for some judges, are those really the judges we want in the tent?

Boshizzle
10-27-2011, 08:44 PM
I'm tired of spending my hard earned money on a contest only to be thrown out of the top 5 by that dang judge #5...(its always that guy).


How do the top five feel about that dang judge #5? Seriously, it's not always the judges. The teams that win on a particular day are happy with the judges. The teams that don't aren't.

I think that judges need to be reminded that a KCBS comp is a meat contest. It's not about the sauce or the garnish or whether or not there is sauce on the lid of the box.

I also think that a more rigorous training program for table captains along with giving table captains some avenue of being able to correct bad habits practiced by judges early on instead of just having to bite their tongues throughout all four categories while some judge at the table continually makes bad judgements based on something other than the rules.