KCBS Juding and Ways to Improve It

DISCLAIMER: For those that read this on the KCBS Board please do not take these comments as we hate the system as many of us do not. We just want to make the KCBS the best BBQ circuit out there. So as members we would like to voice some of our concerns and provide feedback on improvements.

Click to expand...

After reading Gordon's post over here about the issues he experienced at the American Royal - I thought I would start this thread on ways in which the KCBS can improve judging (this was originally posted over at The Smoke Ring, but I thought I would get everyone's comments here as well).

First off - I think at a bare minimum we need to start tracking the scoring of judges. This is without a doubt the biggest step we can make to working with judges to make their scores more uniform across the chart.

Second - Going one step further I think a simple review after the table has finished judging the box on why everyone scored they way they did would be great. I am sure we can come up with a simple piece of paper that has the standardized issues with boxes/meats. For example, on a brisket box a judge could not like the brisket because it was too tough, dry, bad seasoning, cut too big, etc...
If we had this simple sheet we could better understand from the cooks side why it was scored the way it was.

Third - (and this could be a radical move, so do not crucify me) nobody is a judge if they are not a CBJ. Stop bring people off the street. Is it going to result in judges having multiple rounds of boxes, yes - but at least we have certified judges judging the competitions.

I know others have some ideas on things that can be improved so please post them.

Slamdunkpro said:

Give the cooks meaningful feedback - instruct judges to use terms that people can relate to. “Terrible” and “Wonderful” are descriptive terms but hardly quantifiable or helpful. On the other hand “Tough” “Dry” “Salty” “Sweet” “Tender” “Moist” and so on not only give feedback cooks can understand they can also help judges when adjudicating a score.

My 2 cents.

Click to expand...

Perfectly said - it helps cooks understand what was wrong with it as well as judges "recap" why they gave it it a 6 instead of 8 or 9.

As I took the KCBS Judging class a few weeks back, I was thinking to myself "wouldn't it be great if there was some consistency in scoring that made it less subjective and more descriptive to the cook?" Since KCBS taught me to "start at a 6 and go up or down from there," how about quantifiable adders/subtractors so that you'd arrive at a score (based upon your tastes) with more feedback toward the team? Something like a list of plusses or minuses with checkboxes.... Start at 6 then subtract one for "too salty" or subtract 2 for "way too salty." Or add one for "just the right amount of heat" and add one for "just the right amount of sweet." As mentioned, things like "tastes great," and "wonderful" don't do anything. Let's start measuring the dimensions of taste and texture, and provide quantifiable reasons for going up or down the scale.

My $0.02.

Aaron
8 Ball and the 'Que Sticks

jvreeland said:

After reading Gordon's post over here about the issues he experienced at the American Royal - I thought I would start this thread on ways in which the KCBS can improve judging (this was originally posted over at The Smoke Ring, but I thought I would get everyone's comments here as well).

First off - I think at a bare minimum we need to start tracking the scoring of judges. This is without a doubt the biggest step we can make to working with judges to make their scores more uniform across the chart.

Second - Going one step further I think a simple review after the table has finished judging the box on why everyone scored they way they did would be great. I am sure we can come up with a simple piece of paper that has the standardized issues with boxes/meats. For example, on a brisket box a judge could not like the brisket because it was too tough, dry, bad seasoning, cut too big, etc...
If we had this simple sheet we could better understand from the cooks side why it was scored the way it was.

Third - (and this could be a radical move, so do not crucify me) nobody is a judge if they are not a CBJ. Stop bring people off the street. Is it going to result in judges having multiple rounds of boxes, yes - but at least we have certified judges judging the competitions.

I know others have some ideas on things that can be improved so please post them.

Click to expand...

As this is the Brethren Forum I will let Gordon do his talking over on that forum and address your comments here. I am a cook, an organizer and a judge. I judged both the AR Invitational, Open sides (all three categories), open desserts, and the all of the Open meats.

If you have ever stopped and watch what the Reps and most table captains are doing while judges are stuffing food in their mouths your questions would be answered. Every time I have judged, the table captain has gone over each judges slip before handing them to the Rep. At that point every Rep I have had has gone over the slips again before handing them to be tallied. So judges are being tracked and if needed they are being called out if a Rep needs to do so. This was explained at both the invitaional and the Open. It happens.

I have not judged a comp where the judges did not go over the scoring and the reasons why they are judged. There is no way a standardized form would be able to be made let alone the time to go over things and mark them as needed. I will use your brisket example. I was talking with two guys and one said he prefferd his brisket cut thinner than the standard number 2 pencil as laid out by KCBS the other guy said he liked his much thicker than the # 2 Pencil. These two guys were not judges however they were 2 guys from the SAME TEAM cooking the open. So a standardization in the judges tent would be tough. Turn the card over and write why if you want, it is encouraged and I do it If I have to give something 5 or below.

All CBJ's would be wonderful. But often time a non CBJ is not as harsh as a CBJ so better scoring without all CBJ's. Not right, but there it is. Teams go out of their way to see how the judges tent will be stacked so they can adjust their cooking styles.

When I cook I know if my product sucks or not and when I get a 5 or 6 I know exactly why. To many cooks are to quick to look at the judges tent and not at themselves.

Now how to improve? How about reinstating the cooking regulation for the judge that is going to be given his Master Badge. I think this was a very poor decision on the part of KCBS. I actually think a judge should cook with a team LONG before 30 judging events. It is sad how clueless many judges are about what it takes and what goes on in the pit. So as a team, if you are not already a CBJ get to a class. Once there listen and ask questions. If you are team have a CBJ cook with you one weekend and show him the ropes, I guarantee he or she will be a better judge.

Sorry rant over.

smoke-n-my-i's said:

I think the reason there is so much controversy is the way some of the scores we get. For example, on one of my scores, I had 4,4,6,8,8, and a 9. So why such a difference in the scores for the same piece of meat???? Why two 4's with two 8's and a 9?????? That tells me that something is astray.... Bill

Click to expand...

You can never tell someone what they like or dislike. People have their own tastes...that can never change.
I judged a comp. a few years ago and one rib had (IMHO) way too much cumin. Its all I could taste for the rest of the rib entry's. When rib judging was done we were all talking about the ribs and one guy said 'that was the best rib I ever had!" He was saying that about the rib I hated! How can that ever change? I think as cooks we need to know where the lines are. It took me just over a year to find out whats 'too hot'

Mike
Lakeside Smokers
NEBS
KCBS
CBJ

I have noticed the differences in scoring this year as both a judge and team. I think that a lot of judges don't understand that what they are judging isn't supposed to be like commercial BBQ. I believe that some think that it should be and score accordingly when it's not.
I also think that maybe before a judge can judge more than 2 or 3 contests that they should have to cook with a team. That way they understand what goes on on the other side.
What about having a mentor or two at the judging to help new judges in scoring. This would be helpful where a contest is a first time contest or where there was a recent judging class. I know in my hometown they had a judging class that had over 100 people attend and they were given first choice to judge this contest. That is just too many new judges. The scores were crazy.
I think for the most part judges do a good job, but I think that a little support after the class wouldn't hurt.

smoke-n-my-i's said:

I think the reason there is so much controversy is the way some of the scores we get. For example, on one of my scores, I had 4,4,6,8,8, and a 9. So why such a difference in the scores for the same piece of meat???? Why two 4's with two 8's and a 9?????? That tells me that something is astray....
Bill

Click to expand...

The KCBS scoring is amazingly consistent from a big picture perspective. For the most part, the same teams generally win and the same teams generally do poorly. If the scoring model didn't work, we wouldn't have teams that get and stay hot.

If your entries get inconsistent scores in more than 1 contest it's time for a change. I'd recommend a change after a single poor showing. The top teams don't get 4s or 6s on any entry at any contest so something has to change if you're trying to win. Somehow you'll have to find out how to broaden the scope of the audience that likes your BBQ. To help determine what needs to change, post a pic of your entry along with your recipe and preparation techniques on this site and I'll bet many Bretheren will give you pointers without needing to taste your food.

I've always had a problem with how BBQ is judged. Mostly because of the lack of feedback in the process. I'd love to have some feedback even if it was to confirm my own suspicions (tough, salty, sweet, are you trying to poison me?)

I am a Beer Judge for Homebrew (and yes, I am a homebrewer.) The program associated with training and tracking judges has alot in common with the CBJ program. One of the things I like about BJCP is the requirement to try and give feedback to the brewer. Even if I find a beer lacking (and trust me I have both sides of really bad beer and most excellent beer) I try to give some feedback saying why I personally don't like it.

Just like in BBQ judging, there is a table captain to review the scoresheets as they come in to make sure a judge doesn't slack off, or misunderstand the process. If the table captain sees something odd, it's their responsibility to ask the judge "are you sure you want to score this way?" In some cases, the judge will say "yep, I have my reasons".

There is a parallel in beer and bbq, both can be very cheap hobbies, and very expensive hobbies. Time, materials, and equipment can get pretty pricey quick. That said, serious competitors should at least have some better feedback than what we get now.

I know some people will say "there isn't time to give feed back!" I say BS, a BBQ tasting is 6-7 containers, and all you have to do is circle a couple of numbers, and on occasion fill out the comments. On the other hand, When I judge a flight of beers (a specific category), I may be judging up to 14 different beers. I still find the time to fill out the scoresheet completely with the intent on giving the brewer meaningful feedback.

Here's the scoresheet you use for beer judging..

http://www.bjcp.org/SCP_BeerScoreSheet.pdf

One of the things you will notice on the left is checkboxes for common flaws associated with beer. I don't see why these also can't be incorporated into the bbq judging process. I've head there is an attempt at something like this via KCBS. Other things you notice is every scoresheet has your name, BJCP number if you are a BJCP judge, and the score "spreads" to make sure you are not scoring incorrectly.

I realize that you have to have faith in the scoring system for the system to work. But the scoring system used by KCBS as the gold standard is weak IMHO. Considering all the monetary and personal tolls we shell out in order to compete, there should be some feedback to guide us into making great Q.

Anyhow, i'm done ranting. Sorry if I have offended anyone. But I am of the opinion you can have a great time making awesome Q and still improve as you go along. Providing you have the right feedback from peers, quality judges, and trial and error.

Last thing, was anyone else put off by the smell of the livestock near the turnin at the royal? Did the judges have to smell manure while judging? That would put my appetite and sense of taste all out of whack.